Network Ports: Difference between revisions

From OSNEXUS Online Documentation Site
Jump to navigation Jump to search
mNo edit summary
m osn-seo-utilities: docs-arp-flux-protection-for-multi-homed-ha @ d54c6794ab44 (approved in the portal)
 
(78 intermediate revisions by 2 users not shown)
Line 1: Line 1:
[[Category:admin_guide]]
[[Category:admin_guide]]


== Network Port Configuration ==
Network ports (also called target ports or network interfaces) are the Ethernet interfaces through which a QuantaStor system is managed and through which it serves storage. This page covers how to address a port, how to combine ports into a bond, how to layer VLAN and virtual interfaces on top of a port, how to control which protocols each port carries, how QuantaStor keeps ARP answers on the right port, how to add static routes, and which TCP and UDP ports and external sites QuantaStor needs so you can lock the network down without breaking it.
Network Ports (also referred to as Target Ports and Network Interfaces) represent the Ethernet interfaces through which your appliance is managed and storage is accessed. Client hosts (aka initiators) access SAN storage ''Storage Volumes'' (aka targets) via the iSCSI protocol via network ports and NAS file storage which are referred to as ''Network Shares'' in QuantaStor are accessible via the NFS and SMB/CIFS protocols via network ports, and finally object storage is accessible via network ports.


[[File:Modify Network Port - menu.jpg | thumb | 926px | To modify the network port right click on the port in question from either left or middle pane. Alternatively left click on ''Modify'' in the Network Port ribbon at the top of the screen.]]
Client hosts reach [[Storage Volumes]] over iSCSI and NVMe-oF/TCP through a network port, authorized users reach [[Network Shares]] over NFS and SMB through a network port, and object storage buckets are reached over S3 through a network port. Every one of those paths depends on the port configuration described here, so it is worth getting right before you provision storage.


=== Static vs Dynamic (DHCP) Assigned IP Addresses ===
{| class="wikitable"
! Section !! Purpose
|-
| [[#Port types and naming|Port types and naming]] || How to tell a physical port, a bond, a VLAN, a VIF and a cluster VIF apart by name.
|-
| [[#Where network ports are managed|Where network ports are managed]] || The tabs, toolbar and right-click menu that hold every port operation.
|-
| [[#Static vs DHCP assigned IP addresses|Static vs DHCP assigned IP addresses]] || Why static addressing is required in practice, and the one-time conversion on a fresh install.
|-
| [[#Modifying network port settings|Modifying network port settings]] || Every field on the General Settings tab of Modify Network Port.
|-
| [[#Firewall settings per port|Firewall settings per port]] || Allowing and blocking individual protocols on one port.
|-
| [[#NIC bonding and trunking|NIC bonding and trunking]] || Combining ports for throughput and fault tolerance, and which bonding mode to pick.
|-
| [[#VLAN interfaces|VLAN interfaces]] || Tagged virtual ports on a physical or bonded port.
|-
| [[#Virtual interfaces|Virtual interfaces]] || Extra IP addresses on one port.
|-
| [[#High-availability and cluster virtual interfaces|High-availability and cluster virtual interfaces]] || Floating IPs that move with a pool, a site or the grid.
|-
| [[#ARP Flux Protection|ARP Flux Protection]] || How QuantaStor stops a multi-homed system answering ARP on the wrong port, and the ARP Filtering policy.
|-
| [[#Renaming a network port|Renaming a network port]] || Giving a port a stable, meaningful name.
|-
| [[#Online, offline, restart and rescan|Online, offline, restart and rescan]] || Bringing a port up or down and picking up out-of-band changes.
|-
| [[#Static route management|Static route management]] || Routing specific networks through a gateway other than the default.
|-
| [[#Choosing which ports carry which traffic|Choosing which ports carry which traffic]] || Separating management traffic from storage traffic.
|-
| [[#TCP and UDP ports QuantaStor uses|TCP and UDP ports QuantaStor uses]] || Every port the appliance listens on, why, and which ones the Firewall tab can block.
|-
| [[#External sites QuantaStor connects to|External sites QuantaStor connects to]] || Outbound connections for licensing, upgrades, support logs, alerting and replication.
|-
| [[#Locking down the network|Locking down the network]] || What you can safely block, and what has to stay open.
|-
| [[#Verifying connectivity|Verifying connectivity]] || The Network Check report.
|-
| [[#Command line reference|Command line reference]] || The <code>qs</code> commands that cover everything on this page.
|}


QuantaStor supports both statically assigned IP addresses as well as dynamically assigned (DHCP) addresses.  After the initial installation of QuantaStor the eth0 port is typically configured via DHCP with the rest typically disabled. We recommend that one reconfigure all ports with static IP addresses unless the DHCP server is setup to deliver statically assigned addresses via DHCP as identified by MAC address.  Network ports setup with DHCP assigned IP addresses run the risk of changing which can lead to outages.
== Port types and naming ==


=== Modifying Network Port Settings ===
QuantaStor names every port after the interface the kernel presents, and uses two delimiters to indicate that a port is layered on top of another one. Reading the name tells you what a port is:


To modify the configuration of a network port first select the ''Storage System'' section under the main ''Storage Management'' tab then select the ''Network Ports'' section center of the screen.   This section shows the list of all network ports on each appliance. To modify the configuration of a port, right-click on it and choose ''Modify Network Port...'' from the pop-up menu. Alternatively press the "Modify" button in the toolbar at the top of the screen in the "Network Ports" section.
{| class="wikitable"
! Name !! Type !! Notes
|-
| <code>ens192</code>, <code>eno1</code>, <code>enp3s0f0</code> || Physical port || The predictable name assigned by the kernel. Rename it if you want something more meaningful -- see [[#Renaming a network port|Renaming a network port]].
|-
| <code>bond0</code> || Bonded port || Two or more physical ports combined. The member ports become slaves and stop carrying their own IP address.
|-
| <code>ens192.56</code> || VLAN interface || A period separates the parent port name from the VLAN tag. A VLAN on a bond looks like <code>bond0.56</code>.
|-
| <code>ens192:1</code> || Virtual interface (VIF) || A colon followed by an index, allocated from 1 upwards. An extra IP address on the parent port.
|-
| <code>ens192:ha345680</code> || HA pool virtual interface || Floats with a [[Storage Pools|storage pool]] when the pool fails over.
|-
| <code>ens192:sv...</code> || Site cluster virtual interface || Floats between the appliances of a site cluster.
|-
| <code>ens192:gm</code> || Grid management virtual interface || The single floating IP a storage grid is managed through. iSCSI and NVMe-oF are permanently disabled on it, because a management IP that moves on failover is not a safe place to export block storage from.
|-
| <code>ens192 (S)</code> || Manually configured secondary port || An address that QuantaStor discovered but does not own. Management operations are refused on these.
|}


Once the "Modify Network Port" dialog appears select the ''static'' port type and enter the IP address for the port, network mask, and network gateway for the port.  The default MTU should be set to 9000 for all 10GbE and faster network interfaces. If the MTU is changed from the default 1500, be sure that the switch and host side MTU configuration settings are setup to match that of the configured MTU of the QuantaStor appliance ports for the matching network.
Interface names are limited to 15 characters by the kernel, and the tag QuantaStor appends when it creates a cluster virtual interface has to fit inside that budget. A long parent port name is therefore not merely untidy -- it will eventually block cluster VIF creation, so keep renamed ports short.


[[File:Modify Network Port Menu.jpg | 926px]]
== Where network ports are managed ==


=== NIC Bonding / Trunking Configuration ===
[[File:nwport_ports_tab.png|thumb|right|800px|The Network Ports tab lists every port on every system in the grid, grouped by storage system. The Network Port toolbar group above it holds the create, delete and state operations.]]


QuantaStor supports NIC bonding, also called trunking, which allows one to combine multiple ports together to improve network performance and network fault-tolerance.  For all NIC bonding modes except LACP, make sure that all physical ports which are part of the bond are connected to the same network switch.  LACP mode will work across switches but it is important to make sure that the switch ports are configured properly to enable LACP.
{{Navigation|Storage Management &rarr; Storage Systems &rarr; ''select a Storage System'' &rarr; Network Ports ''(tab)''}}


QuantaStor uses a round-robin LACP bonding policy by default but the default is easy to change in the ''Modify Storage System..'' dialog. Round-robin mode provides load balancing and fault tolerance by transmitting packets in sequential order from the first available interface through the last but it doesn't span switches.  For optimal fault tolerance LACP 802.3ad Dynamic Link aggregation mode is required as it will span switches ensuring network availability to the bonded port in the event of a switch outage.
Select the '''Storage Systems''' section in the tree on the left, then the '''Network Ports''' tab in the centre of the screen. The tab lists every port on every system in the grid, grouped by storage system, with its state, link state, link speed, IP address, subnet mask, whether it is an iSCSI or NVMe-oF portal, and its vendor, model, MAC address, byte counters and MTU. The '''Network Static Routes''' tab beside it lists the static routes on those same ports.


=== VLAN Configuration ===
The '''Network Port''' toolbar group at the top of the screen carries Modify, Create Virtual Port, Create VLAN Port, Create Bonded Port, the matching Delete operations, Online, Offline and Rescan All.


QuantaStor supports the creation of VLAN ports which are simply virtual ports with VLAN tagging and QoS controls. To create a VLAN port simply right-click on a network port and choose the ''Create VLAN Interface..'' option as shown in the pop-up menu shown at the top of this section.  VLANs can be created and deleted at anytime and are easily identified as they are named with a period (.) as a delimiter between the parent port name and the VLAN tag number.  For example, a VLAN with ID 56 associated with eth1 will have the name eth1.56.  VLANs can also be created on bonded ports which yields ports with names like bond0.56.
[[File:nwport_context_menu.png|thumb|right|237px|Right-clicking a port in the Network Ports tab reaches every port operation, including the three that are not on the toolbar: Rename, Restart, and Create/Delete Static Route.]]


=== Virtual Port Configuration ===
Three operations are reachable only by right-clicking a port, not from the toolbar: '''Rename Network Port...''', '''Restart Network Port...''' and '''Create Static Route...''' / '''Delete Static Route...'''. Right-click a port either in the tree on the left or in the Network Ports tab.


Virtual ports allow one to use a given physical port on multiple networks with multiple IP addresses simultaneously.  To create a virtual port on a physical port simply right-click on it and choose ''Create Virtual Interface..'' from the menu.  Note that virtual ports (aka virtual interfaces) can only be attached to physical ports or bonded ports and not other virtual ports or VLAN ports.  Virtual ports can be created and deleted at anytime.
For grid-wide settings such as the DNS servers and search domain that apply to all ports, use '''Modify Grid Network Settings''' instead -- the '''Modify Grid''' button in the '''Storage System Grid''' toolbar group.  It writes one DNS search suffix, DNS server list and NTP server list to every system you tick, overwriting whatever those systems had.  The per-system equivalents are the [[Storage System#DNS Servers|DNS Servers]] and [[Storage System#NTP Servers|NTP Servers]] tabs of Storage System Modify.


=== High-availability Virtual Ports/Interfaces ===
== Static vs DHCP assigned IP addresses ==
QuantaStor also has some special virtual ports which are used in High-Availability configurations. You'll see these ports with :ha in the name such as eth0:ha345680.  These HA virtual interfaces are associated with specific storage pools and move with the pool when the pool is failed-over to another appliance.  Pools can have many VIPs (virtual ports) assigned to them and as more are assigned the last digit of the port name is incremented from 0, 1, 2, 3, 4 and so on.


HA Virtual ports are created by right-clicking on an HA Failover Group in the Storage Pool section and then choosing the ''Create Virtual Interface..'' option.  They can also be created via the ''Cluster Resource Management'' section. Note also that a cluster heartbeat (aka Site Cluster) must be setup between the appliances used to manage a given HA pool before HA VIFs/VIPs may be created.  See more on HA cluster setup in the walk-thru guide found here.
QuantaStor supports both statically assigned IP addresses and dynamically assigned '''D'''ynamic '''H'''ost '''C'''onfiguration '''P'''rotocol ('''DHCP''') addresses. After installation the first port is typically configured by DHCP and the rest are disabled, so that the appliance is reachable for initial setup.


=== Configuring Network Management Ports===
Reconfigure every port with a static IP address before you provision storage. A DHCP lease can change, and a storage system whose address moves takes its iSCSI targets, NFS exports and SMB shares with it. The appliance also has to be able to boot and serve storage when the DHCP server is unreachable. The only defensible use of DHCP is a reservation that always hands the same address to the same MAC, and even then a static address is simpler to reason about.


QuantaStor works with all major 10/25/40/56/100GbE network cards (NICs) from Intel, Mellanox, Cisco, Dell, HPE, SuperMicro, and others. We recommend that one designate slower 1GbE ports as ''iSCSI disabled'' which will prevent their use by iSCSI initiators and relegate them for use only with management traffic. Similarly, one should have the 1GbE ports on a separate network from the ports used for NAS traffic so that performance is not impacted by the system using the 1GbE ports by mistake.
There is a second, one-time reason to set a static management address first. A freshly installed system still has the installer's netplan configuration active. The first time you modify any network port, QuantaStor converts that configuration into {{Code|1=/etc/network/interfaces}}, backs up the previous file as {{Code|1=/etc/network/interfaces-netplan-convert-<timestamp>.backup}}, renames each {{Code|1=/etc/netplan/*.yaml}} file to <code>.bak</code>, and disables the netplan renderer. All interfaces are torn down and brought back during that conversion, so a port still on DHCP can come back with a different address and the management session drops. QuantaStor guards against this: while any port is in that state it is flagged with a warning whose detail begins "Netplan", and the Modify Network Port dialog refuses any change other than to <code>static</code>, with the message ''"Be sure to configure a static management network port before modifying network ports."''
 
Set a static address on your management port first, then work through the remaining DHCP ports until they all have static IPs assigned. '''Reboot afterwards and confirm the configuration, which is what the accompanying alert asks for.'''  This is an important step after an initial QuantaStor OS installation as this allows the system to cleanup network configuration issues left behind by the text mode installer.
 
== Modifying network port settings ==
 
[[File:nwport_modify_general.png|thumb|right|550px|The General Settings tab of Modify Network Port. The Static IP Configuration Settings fieldset is only enabled when Config Type is set to static.]]
 
{{Navigation|Storage Management &rarr; Storage Systems &rarr; Network Port ''(select + right-click)'' &rarr; Modify Network Port...}}
 
The dialog is also on the '''Modify''' button in the Network Port toolbar group. It has two tabs, '''General Settings''' and '''Firewall'''.
 
=== General Settings ===
 
{| class="wikitable"
! Field !! Description
|-
| '''Storage System''' || The system whose port you are configuring. Changing it reloads the Network Port list.
|-
| '''Network Port''' || The port to modify. Every port on the selected system is offered, including bonds, VLANs and virtual interfaces.
|-
| '''Config Type''' || <code>static</code>, <code>dhcp</code> or <code>disabled</code>. Only <code>static</code> enables the Static IP Configuration Settings fieldset below. Choosing <code>disabled</code> prompts for confirmation and then removes the port's stanza from {{Code|1=/etc/network/interfaces}} entirely, so the port stops carrying traffic and does not come back after a reboot. A port that is a bond slave reports as <code>disabled</code> and cannot be changed here -- delete the bond first. Cluster virtual interfaces are locked to <code>static</code> because the cluster software owns their addresses.
|-
| '''Description''' || An optional note recorded against the port. Use it to record what the port is for; there is no other place to do that.
|-
| '''iSCSI Portal''' || Makes this port's IP address an allowed iSCSI portal. Clearing it removes the address from the allowed-portal list of every iSCSI target on the system, so initiators can no longer log in through it. Existing sessions are not torn down -- the change takes effect on the next login. If no port at all is left iSCSI-enabled, QuantaStor restricts iSCSI to loopback rather than leaving it open.
|-
| '''NVMeoF Portal''' || Makes this port's IP address an NVMe-oF/TCP listener on port 4420. Unlike the iSCSI setting, clearing it removes the listener, which does drop connections through that address.
|-
| '''Lossless Networking''' || Configures RoCEv2 with priority flow control on the port. Only selectable on adapters that support it -- in practice Mellanox/NVIDIA ConnectX cards with RDMA enabled -- and greyed out everywhere else. It sets DSCP-based priority trust, enables PFC on priority 3 only, turns global pause off, sizes the receive buffers for the adapter model, and records the port in {{Code|1=/var/opt/osnexus/quantastor/conf/qs_losslessNetworking.conf}}. QuantaStor verifies the result and reverts the change if verification fails. Enable it only where the switch fabric is configured for the same priority flow control scheme; a one-sided configuration performs worse than plain Ethernet.
|-
| '''Optimize hardware RX/TX buffer settings for throughput''' || Enlarges the adapter's receive and transmit ring buffers. QuantaStor records the current sizes so it can put them back, then raises both rings to the largest power-of-two multiple of 128 that still fits under the adapter's hardware maximum -- roughly half the maximum, which improves large-block and replication throughput without asking the driver for a size it will reject. Settings are recorded in {{Code|1=/var/opt/osnexus/quantastor/conf/qs_porttunings.conf}} and re-applied every five minutes. Available on physical ports only: a bond, VLAN or virtual interface has no hardware ring of its own, so enable it on the member ports instead.
|}
 
The '''Static IP Configuration Settings''' fieldset holds the addressing:
 
{| class="wikitable"
! Field !! Description
|-
| '''IP Address''' || The static address for the port. Each port needs a unique address; QuantaStor rejects an address already recorded on another port. If the address is part of a Ceph cluster configuration you have to disable the port before the address can be changed.
|-
| '''Subnet Mask''' || The mask for the port's network, such as <code>255.255.0.0</code>.
|-
| '''Gateway''' || Optional. QuantaStor checks that the gateway is on the same subnet as the port address and refuses it otherwise. Configure a gateway on one port only. QuantaStor gives every static port the same route metric, so a second gateway-bearing port ends up competing for the default route rather than adding a fallback.
|-
| '''MTU''' and '''Jumbo Frames''' || The maximum transmission unit, 1492 to 64000. The '''Jumbo Frames''' button toggles between 9000 and 1500 and relabels itself '''Default Frames''' once jumbo is set. Set 9000 on all 10GbE and faster interfaces. If you move away from 1500, the switch ports and the client hosts on that network have to match -- a mismatched MTU produces intermittent stalls rather than a clean failure. The field is disabled for VLAN and virtual interfaces, which inherit the parent port's MTU; changing the parent cascades the new MTU to all of its children. Setting a VLAN MTU above its parent's is capped to the parent's value with an alert.
|}
 
Changing the IP address of a port that has static routes configured is flagged: the routes are bound to the old address and client connections through them can drop. Remove the routes first.
 
Click '''Apply''' to commit the change and keep the dialog open, or '''OK''' to commit and close.
 
== Firewall settings per port ==
 
[[File:nwport_modify_firewall.png|thumb|right|550px|The Firewall tab sets each protocol to Allow, Block or Inherit for this port only.]]
 
{{Navigation|Storage Management &rarr; Storage Systems &rarr; Network Port ''(select + right-click)'' &rarr; Modify Network Port... &rarr; Firewall ''(tab)''}}
 
The '''Firewall''' tab lists each service QuantaStor knows how to filter, with its description and a '''Status''' setting. The three settings are what makes this useful, and the distinction is easy to miss:
 
* '''Inherit''' -- follow the system-wide firewall setting. This is the default for every service on every port.
* '''Block''' -- drop traffic for this service arriving at this port's IP address, even though the system-wide setting allows it.
* '''Allow''' -- accept traffic for this service at this port's IP address, even though the system-wide setting blocks it.
 
So the system level sets the policy and the port level carries the exceptions. The usual pattern is to block everything you do not use at the system level -- see [[Firewall Management]] -- and then Allow the handful of protocols each port is actually meant to serve.
 
Two things are worth knowing before you rely on it. The rules match on the port's '''destination IP address''', not on the interface, so a per-port rule really follows the address; that is deliberate, and it is why the masks are also recorded against HA and site virtual interfaces so the rule follows a floating address to whichever system holds it. And because QuantaStor reports a service as blocked whenever any DROP rule exists for it, blocking a protocol on one port raises an alert saying that service is blocked system-wide, even though only that one address is affected.
 
The service list, the TCP and UDP ports behind each entry, and the bit each one occupies come from {{Code|1=/opt/osnexus/quantastor/conf/qs_services_firewall.conf}} on the appliance. The rules themselves are iptables rules in QuantaStor's own chains (<code>QUANTASTOR-INPUT</code>, <code>QUANTASTOR-FORWARD</code>, <code>QUANTASTOR-OUTPUT</code>), written out to {{Code|1=/var/opt/osnexus/quantastor/iptables.qsfirewallrules}} so they survive a restart, and kept out of the shared chains so the firewall can be rebuilt without disturbing other rules on the system. QuantaStor re-applies them on change and again every 20 minutes.
 
Changing a setting that makes a service unreachable prompts for confirmation first. For the TCP and UDP ports behind each entry, and the ports the Firewall tab does not cover, see [[#TCP and UDP ports QuantaStor uses|TCP and UDP ports QuantaStor uses]] below.
 
The same firewall settings can be driven from the CLI with <code>[[QuantaStor CLI Command Reference#network-port-modify|qs network-port-modify]] --protocol-block=</code> and <code>--protocol-allow=</code>, each taking a comma-separated protocol list, with <code>--protocol-block-reset</code> and <code>--protocol-allow-reset</code> to clear a list. Prefix a protocol with a tilde (<code>~</code>) to remove it from a list rather than add it.
 
== NIC bonding and trunking ==
 
[[File:nwport_bond_create.png|thumb|right|550px|Create Bonded Port. At least two ports must be selected in the grid; clear Hide Configured Ports to bond a port that already has an IP address.]]
 
{{Navigation|Storage Management &rarr; Storage Systems &rarr; Network Port &rarr; Create Bonded Port ''(toolbar)''}}
 
Bonding, also called trunking or teaming, combines several physical ports into one logical port to improve throughput and to survive the loss of a cable, a switch port or a NIC. The bond takes the IP address; the member ports become slaves and no longer carry addresses of their own.
 
For every mode except LACP, all of the member ports must be connected to the same switch. LACP works across switches -- which is what makes it the only mode that survives a whole switch going away -- but it requires the switch ports to be configured for LACP as well. QuantaStor surfaces switch-side mistakes on the port itself, reporting an invalid partner MAC address or a partner that is not advertising LACP support, so check the port state after creating an LACP bond rather than assuming it came up.
 
The dialog fields are the same addressing fields as Modify Network Port, plus:
 
{| class="wikitable"
! Field !! Description
|-
| '''Bond Mode''' || The bonding policy. See the table below.
|-
| '''Select the Network Ports to be Bonded''' || Select at least two ports. The grid lists each candidate port's name, IP address, vendor and model.
|-
| '''Hide Configured Ports''' || Selected by default, hiding ports that already have an IP address so you do not accidentally bond your management port. Clear it to bond a configured port; QuantaStor warns that the port's existing IP configuration will be removed and that the bond will only have the address you entered.
|-
| '''Force''' || Proceeds without the confirmation prompt when a selected port already has an IP configuration.
|}
 
The modes QuantaStor offers, with the switch requirement each one carries:
 
{| class="wikitable"
! Mode !! Kernel mode !! Switch !! Behaviour
|-
| Link Aggr Ctrl Protocol (LACP layer2) || <code>802.3ad</code>, hash <code>layer2</code> || Managed, LACP configured || Balances by source and destination MAC. Spans switches. Recommended default.
|-
| Link Aggr Ctrl Protocol (LACP layer2+3) || <code>802.3ad</code>, hash <code>layer2+3</code> || Managed, LACP configured || Adds the IP addresses to the hash, which spreads traffic better when many clients sit behind one router MAC.
|-
| Link Aggr Ctrl Protocol (LACP layer3+4) || <code>802.3ad</code>, hash <code>layer3+4</code> || Managed, LACP configured || Adds the TCP/UDP ports to the hash, so a single client with many connections can use more than one member link.
|-
| Round Robin (balance-rr) || <code>balance-rr</code> || Etherchannel, managed || Sends packets in sequence across the members. Highest single-stream throughput, but it reorders packets and does not span switches.
|-
| Adaptive Transmit Load Balancing (balance-tlb) || <code>balance-tlb</code> || Unmanaged || Balances outbound traffic only; inbound arrives on one member. Needs no switch configuration.
|-
| Adaptive Load Balancing (balance-alb) || <code>balance-alb</code> || Unmanaged || As balance-tlb, plus inbound balancing via ARP negotiation. Needs no switch configuration.
|-
| Balance XOR (balance-xor) || <code>balance-xor</code> || Etherchannel, managed || Picks a member by hashing the MAC addresses. Static, so both ends must agree.
|-
| Active-Backup (active-backup) || <code>active-backup</code> || Unmanaged || One member carries all traffic; the others stand by. No throughput gain, and the simplest mode to get right.
|}
 
For fault tolerance that survives a switch outage, LACP is the mode to use. Where the switches are unmanaged and cannot be configured, use Active-Backup or one of the adaptive load balancing modes.
 
The '''Bond Mode''' combo in the Create Bonded Port dialog is pre-selected from the storage system's own default bonding policy, which QuantaStor sets on first startup and records in {{Code|1=/etc/modprobe.d/bonding.conf}}. On the 6.9.0 system this page was verified against that default was LACP layer 2. Check yours with <code>[[QuantaStor CLI Command Reference#system-get|qs system-get]]</code> and look at the '''Bond Mode''' field, and change it with <code>[[QuantaStor CLI Command Reference#system-modify|qs system-modify]] --bond-mode=&lt;mode&gt;</code>. QuantaStor sets <code>miimon=100</code> on every bond so link failures are detected within a tenth of a second, and allows a maximum of four bonds per system.
 
To change the mode of a bond that already exists, open '''Modify Network Port''' on the bond and use the '''Bond Port Advanced Settings...''' button, which appears only when the selected port is a bond. The mode cannot be changed while the bond is carrying a virtual interface or an HA virtual interface -- remove those first. If a member port drops out of a bond, QuantaStor re-enslaves it automatically, at most once an hour per member.
 
Delete a bond with '''Delete Bonded Port''' in the toolbar, or <code>[[QuantaStor CLI Command Reference#bonded-interface-delete|qs bond-delete]] --port=&lt;bond&gt;</code>. The member ports return to being unconfigured physical ports.
 
== VLAN interfaces ==
 
[[File:nwport_vlan_create.png|thumb|right|396px|Create VLAN Interface. The MTU is read-only because a VLAN inherits it from the parent port.]]
 
{{Navigation|Storage Management &rarr; Storage Systems &rarr; Network Port &rarr; Create VLAN Port ''(toolbar)''}}
 
A '''VLAN''' ('''V'''irtual '''LAN''') interface is a virtual port that tags its traffic with a VLAN ID, letting one physical or bonded port serve several isolated networks. VLAN interfaces can be created and deleted at any time.
 
{| class="wikitable"
! Field !! Description
|-
| '''Storage System''' || The system to create the VLAN interface on.
|-
| '''Parent Port''' || The physical or bonded port to tag on. A VLAN cannot be created on another VLAN, nor on a virtual interface, so those are filtered out of the list.
|-
| '''IP Address''', '''Subnet Mask''', '''Gateway''' || The addressing for the VLAN interface, on the network that VLAN reaches.
|-
| '''Description''' || An optional note. Recording which network the tag corresponds to pays for itself.
|-
| '''MTU''' || Read-only. A VLAN inherits the parent port's MTU; change it on the parent.
|-
| '''VLAN ID''' || The 802.1Q tag, 1 to 4094 (4095 is reserved). Defaults to 5.
|-
| '''VLAN QoS''' || The 802.1p Class of Service user priority to record for the interface, 0 (lowest, the default) to 7 (highest).
|}
 
The new port is named after its parent with a period and the tag, so VLAN 56 on <code>ens192</code> becomes <code>ens192.56</code> and VLAN 56 on <code>bond0</code> becomes <code>bond0.56</code>. Delete a VLAN interface with '''Delete VLAN Port''' in the toolbar or <code>[[QuantaStor CLI Command Reference#vlan-interface-delete|qs vlan-delete]] --port=&lt;port&gt;</code>. A port that has a VLAN child cannot be disabled or renamed until the VLAN is removed.
 
== Virtual interfaces ==
 
[[File:nwport_vif_create.png|thumb|right|500px|Create Virtual Interface. A VIF adds a second address to an existing port.]]
 
{{Navigation|Storage Management &rarr; Storage Systems &rarr; Network Port &rarr; Create Virtual Port ''(toolbar)''}}
 
A virtual interface (VIF) gives a physical or bonded port an additional IP address, so one port can be present on several networks or carry several addresses on the same network. VIFs can be created and deleted at any time, and cannot be layered on top of another VIF.
 
The fields are '''Storage System''', '''Parent Port''', '''IP Address''', '''Description''', '''Subnet Mask''', '''Gateway''', and a read-only '''MTU''' inherited from the parent port. QuantaStor names the new interface after the parent with a colon and the next free index, so the first VIF on <code>ens192</code> is <code>ens192:1</code>.
 
Two constraints are not obvious from the dialog. A VIF cannot be created on a bond running in Active-Backup mode. And DHCP is not valid on a virtual interface -- a VIF is always statically addressed.
 
Delete a VIF with '''Delete Virtual Port''' in the toolbar or <code>[[QuantaStor CLI Command Reference#virtual-interface-delete|qs vif-delete]] --port=&lt;port&gt;</code>.
 
== High-availability and cluster virtual interfaces ==
 
QuantaStor also has virtual interfaces that float between systems. You will see these with <code>:ha</code>, <code>:sv</code> or <code>:gm</code> in the name, such as <code>ens192:ha345680</code>.
 
* '''HA pool virtual interfaces''' (<code>:ha</code>) are associated with a specific [[Storage Pools|storage pool]] and move with the pool when it fails over to another system. A pool can have several; each additional one increments the last digit of the port name, so they run <code>:ha345680</code>, <code>:ha345681</code>, <code>:ha345682</code> and so on. Create them by right-clicking an HA failover group in the Storage Pools section and choosing '''Add Cluster VIF...''', or with <code>[[QuantaStor CLI Command Reference#ha-interface-create|qs hai-create]] --ha-group=&lt;group&gt; --parent-port=&lt;port&gt; --ip-address=&lt;ip&gt;</code>. A cluster heartbeat must be configured between the systems that manage the pool before an HA VIF can be created; see [[Setup Guide for Clustered HA Storage Pools]] and [[Storage Pool HA Failover Interface Create]].
* '''Site cluster virtual interfaces''' (<code>:sv</code>) float between the appliances of a site so that a service address stays available. See [[High-availability VIF Management]].
* The '''grid management virtual interface''' (<code>:gm</code>) is the single floating address a storage grid is managed through. There is exactly one per site cluster, and iSCSI and NVMe-oF are permanently disabled on it.
 
These are all managed from the '''[[High-availability VIF Management]]''' main tab, and by the cluster software rather than by the network port configuration -- that page covers use cases, failover, deliberate moves and location constraints. That is why the Modify Network Port dialog locks their Config Type to <code>static</code>, disables the addressing fields, and refuses IP, MTU and bond mode changes -- and why static routes cannot be created on them. Change a floating address through the HA or site VIF dialogs, not through Modify Network Port.
 
A floating address only works if the switches believe the system that currently holds it, which is what ARP flux protection, below, is for.
 
== ARP Flux Protection ==
 
A system with more than one network port -- every HA node and every Ceph node, and any system with a separate management and storage network -- is '''multi-homed'''. With the Linux kernel's default ARP settings, any port on a multi-homed system will answer an ARP request for any address configured anywhere on the system, and the system may use any of its own addresses as the source of the ARP requests it sends. The switches then learn the wrong MAC address and switch port for an address, and traffic meant for the storage network arrives on the management port, or the other way round. This is known as '''ARP flux'''.
 
ARP flux matters most for the floating addresses described above. When an HA or cluster virtual interface moves, the system taking it over sends a gratuitous ARP so that the switches and clients update their tables. If the switches have already learned the wrong port for that address, or if another port answers for it, clients keep sending to the old location and the failover looks like an outage. On a bond carrying HA virtual interfaces the effect is worse, because the address also has to be learned through the right member link.
 
QuantaStor guards against this with two separate mechanisms. One is always on; the other is the '''ARP Filtering''' policy in Storage System Modify.
 
=== Always-on ARP flux protection ===
 
When the QuantaStor service starts, it sets four kernel parameters:
 
{| class="wikitable"
! Parameter !! Value !! Effect
|-
| <code>net.ipv4.conf.all.arp_ignore</code> || <code>1</code> || Answer an ARP request only when the requested address is configured on the port the request arrived on.
|-
| <code>net.ipv4.conf.default.arp_ignore</code> || <code>1</code> || The same, for ports created after the service started.
|-
| <code>net.ipv4.conf.all.arp_announce</code> || <code>2</code> || Always use the best local address for the destination as the source of an ARP request, never an address that belongs to another port.
|-
| <code>net.ipv4.conf.default.arp_announce</code> || <code>2</code> || The same, for ports created after the service started.
|}
 
The kernel applies the larger of the <code>all</code> value and a port's own value, so the <code>all</code> settings cover every existing port and the <code>default</code> settings cover bonds, VLANs and virtual interfaces created later. QuantaStor applies the values immediately and also writes them to {{Code|1=/etc/sysctl.d/70-quantastor-arp.conf}}, so they survive a reboot and an upgrade takes effect without one. The file is managed by QuantaStor; do not edit it, because QuantaStor rewrites it on the next service start.
 
There is no setting for this in the web interface. It needs none: with these values a port still answers every ARP request for its own addresses, so single-homed systems behave exactly as before.
 
To check the live values on an appliance:
 
<pre style="font-size: smaller">
sysctl net.ipv4.conf.all.arp_ignore net.ipv4.conf.all.arp_announce
cat /etc/sysctl.d/70-quantastor-arp.conf
</pre>
 
==== Turning it off ====
 
Turn ARP flux protection off only if your network design depends on the kernel's default ARP behaviour, for example a host-based load balancing scheme that expects any port to answer for any address. Create the touch file and restart the QuantaStor service:
 
<pre style="font-size: smaller">
touch /var/opt/osnexus/quantastor/touchfiles/tf_arp_flux_protection.disable
systemctl restart quantastor
</pre>
 
On the next start QuantaStor deletes {{Code|1=/etc/sysctl.d/70-quantastor-arp.conf}} and sets all four parameters back to the kernel default of <code>0</code>. It reverts them only if that file exists, so values you set by hand with no QuantaStor file present are left alone. To turn the protection back on, delete the touch file and restart the service again. The setting is per system: on an HA pair or a cluster, do the same on every member, or the members will answer ARP differently. See [[QuantaStor Touch Files]] for touch files in general.
 
Setting <code>disable_arp_logic=true</code> in the <code>[system]</code> section of {{Code|1=/etc/quantastor.conf}} has the same effect, and also stops QuantaStor from managing the ARP Filtering policy below.
 
=== ARP Filtering policy ===
 
[[File:Docs-arp-flux-protection-for-multi-homed-ha-arp-filtering.png|thumb|right|590px|The ARP Filtering fieldset on the Network Settings tab of Storage System Modify. Status shows whether filtering is currently in effect, here Disabled under the Auto policy on a system with no bond or virtual port.]]
 
{{Navigation|Storage Management &rarr; Storage Systems &rarr; ''select a Storage System'' &rarr; Modify ''(toolbar)'' &rarr; Network Settings ''(tab)''}}
 
The '''ARP Filtering''' fieldset on the '''Network Settings''' tab of [[Storage System Modify]] controls a second kernel parameter, <code>arp_filter</code>. With <code>arp_filter</code> on, the kernel answers an ARP request on a port only if it would route traffic for the requester out of that same port. It adds to the always-on protection when two ports sit on the same subnet, which is where <code>arp_ignore</code> alone cannot decide which port should answer.
 
{| class="wikitable"
! Field !! Description
|-
| '''ARP Policy''' || '''Auto''' (the default), '''Enabled''' or '''Disabled'''. '''Auto''' turns filtering on when the system has a bonded port or any other virtual port, and off otherwise. '''Enabled''' and '''Disabled''' force it on or off regardless of the ports.
|-
| '''Status''' || Read-only. Whether ARP filtering is in effect right now, read from the running kernel: '''Enabled''' or '''Disabled'''.
|}
 
QuantaStor writes the setting to {{Code|1=/etc/sysctl.conf}} as <code>net.ipv4.conf.all.arp_filter</code> so it survives a reboot, applies it to the running kernel, and, if the value changed, flushes the system's ARP table so neighbours are relearned through the right ports.
 
Under '''Auto''', QuantaStor re-evaluates the setting when the service starts, when you change the policy, and when a bond is deleted. Creating a bond turns filtering on immediately. Creating a VLAN or virtual interface does not, by itself, re-evaluate the policy; the change is picked up the next time the service starts. If you are building a multi-homed configuration and want filtering on from the first port, set the policy to '''Enabled'''.
 
Leave the policy on '''Auto''' unless you have a specific reason not to. Use '''Enabled''' on a system that has two or more physical ports on the same network without a bond. Use '''Disabled''' only if filtering interferes with a routing design that deliberately sends replies out of a different port from the one the request arrived on.
 
== Renaming a network port ==
 
[[File:nwport_rename.png|thumb|right|550px|Rename Network Port. The Port Information fieldset identifies the port you are about to rename.]]
 
{{Navigation|Storage Management &rarr; Storage Systems &rarr; Network Port ''(select + right-click)'' &rarr; Rename Network Port...}}
 
Predictable kernel names such as <code>enp3s0f1</code> say where the adapter is but nothing about what it does. Renaming a port gives it a name that survives hardware changes and reads clearly in the port list.
 
Enter the new name in '''New Name'''; the '''Port Information''' fieldset below shows the selected port's UUID, IP address, MAC address and state so you can confirm you have the right one. QuantaStor writes a systemd link file at {{Code|1=/etc/systemd/network/10-<newname>.link}} that matches the port by MAC address, renames the running interface, and rewrites the port's stanza in {{Code|1=/etc/network/interfaces}}.
 
The dialog's own guidance is worth following: prefix your management port with <code>mgmt</code>, and avoid the <code>eth</code> prefix, which collides with the names kernel drivers hand out. Interfaces are conventionally numbered from zero, as in <code>eno0</code>, <code>eno1</code>. Keep the name under 15 characters -- see [[#Port types and naming|Port types and naming]]. The name is lowercased for you.
 
A port cannot be renamed while it is a bond slave, while it has VLAN or virtual interface children, or while the system is using legacy <code>ethN</code> naming mode. Remove the bond or the children first, or switch back to predictable naming in [[Storage System Modify|Modify Storage System]]. A reboot may be needed for the rename to take effect everywhere, and QuantaStor raises an alert saying so.
 
The CLI equivalent is <code>[[QuantaStor CLI Command Reference#network-port-rename|qs np-rename]] --port=&lt;port&gt; --name=&lt;newname&gt;</code>.
 
== Online, offline, restart and rescan ==
 
Four state operations sit alongside the configuration dialogs:
 
{| class="wikitable"
! Operation !! Where !! What it does
|-
| '''Online''' || Network Port toolbar; right-click &rarr; Online Network Port... || Brings the port up. The CLI equivalent is <code>[[QuantaStor CLI Command Reference#network-port-enable|qs np-enable]] --port=&lt;port&gt;</code>.
|-
| '''Offline''' || Network Port toolbar; right-click &rarr; Offline Network Port... || Takes the port down without discarding its configuration, so Online brings it back as it was. Static routes on the port go down with it and come back when it is brought back up. The CLI equivalent is <code>[[QuantaStor CLI Command Reference#network-port-disable|qs np-disable]] --port=&lt;port&gt;</code>.
|-
| '''Restart Network Port...''' || Right-click only || Cycles the port so a changed setting takes effect. The CLI equivalent is <code>[[QuantaStor CLI Command Reference#network-port-restart|qs np-restart]] --port=&lt;port&gt;</code>.
|-
| '''Rescan All''' || Network Port toolbar; right-click &rarr; Rescan All Network Ports... || Discovers newly added ports and picks up configuration made outside QuantaStor. The CLI equivalent is <code>[[QuantaStor CLI Command Reference#network-port-rescan|qs np-rescan]]</code>.
|}
 
Offlining a port that has virtual interfaces attached takes the virtual interfaces down with it. Setting Config Type to <code>disabled</code> in Modify Network Port is not the same thing as taking the port offline: offline is temporary, whereas <code>disabled</code> removes the configuration and does not survive a reboot.
 
Ports QuantaStor discovers but that you do not want cluttering the list can be hidden with <code>[[QuantaStor CLI Command Reference#network-port-hide|qs np-hide]] --port=&lt;port&gt; --hide=true</code>.
 
QuantaStor also repairs drift on its own. Discovery re-applies the recorded MTU, subnet mask and gateway when the live configuration diverges from the configuration file, at most once every six hours per correction, and marks the port with a warning telling you to use the modify operation rather than editing the system configuration directly.
 
== Static route management ==
 
A static route customizes traffic flow so that a specific network is reached through a gateway other than the default. This is how you reach a network that the default gateway cannot route to, whether that is across a VLAN, through a VPN tunnel, or over a dedicated replication link. Each route has two parts: the gateway on the local network through which the remote network is reachable, and the address and mask of the remote network itself.
 
QuantaStor's static routes are per port, which gives granular control over which interface carries traffic for which network.
 
=== Creating a static route ===
 
[[File:nwport_static_route_create.png|thumb|right|500px|Create Static Route. The gateway must be on the port's own network; the destination must be given in bind address form.]]
 
{{Navigation|Storage Management &rarr; Storage Systems &rarr; Network Port ''(select + right-click)'' &rarr; Create Static Route...}}
 
The dialog is also reachable by right-clicking inside the '''Network Static Routes''' tab.
 
{| class="wikitable"
! Field !! Description
|-
| '''Storage System''' || The system to add the route to.
|-
| '''Parent Port''' || The port the route is bound to. Floating cluster interfaces are filtered out -- routes cannot be created on them.
|-
| '''Gateway IP Address''' || The gateway on the port's own network through which the destination network is reached.
|-
| '''Destination IP Address''' || The network to route to, in bind address form. QuantaStor checks the address against the mask and rejects anything that is not a network address, telling you the correct form to use.
|-
| '''Destination Subnet Mask''' || The mask of the destination network.
|}
 
For example, on a system whose <code>ens192</code> port is <code>10.0.8.70/24</code> and which has a gateway at <code>10.0.8.1</code> that can reach the <code>192.168.0.0/16</code> network, enter a '''Gateway IP Address''' of <code>10.0.8.1</code>, a '''Destination IP Address''' of <code>192.168.0.0</code> and a '''Destination Subnet Mask''' of <code>255.255.0.0</code>. All traffic from that port destined for <code>192.168.0.0/16</code> is then sent via <code>10.0.8.1</code>. Note that the gateway has to be on the port's own network -- QuantaStor applies the route immediately and refuses a gateway the port cannot reach.
 
Routes are activated the moment they are created, and QuantaStor refuses a gateway it cannot reach rather than recording a route that will never work. A duplicate destination network and mask on the same system is also refused.
 
[[File:nwport_static_routes_tab.png|thumb|right|800px|The Network Static Routes tab lists each route with its parent port, gateway, and destination network and mask.]]
 
Route management is integrated with the platform rather than bolted on. Each route is written to an executable script at {{Code|1=/etc/network/<port>.staticroutes}}, and QuantaStor installs hooks in {{Code|1=/etc/network/if-up.d}} and {{Code|1=/etc/network/if-down.d}} that run it. The routes therefore enable and disable correctly when a port is brought up or down with <code>ifup</code> and <code>ifdown</code> from an SSH session, not just from the web interface. QuantaStor treats those scripts as the source of truth and imports routes it finds in them, so a route added by hand is picked up rather than overwritten. We recommend using the <code>qs</code> CLI or the web management interface all the same, so the routes are recorded as objects you can see in the Network Static Routes tab.
 
The CLI equivalent is <code>[[QuantaStor CLI Command Reference#static-route-create|qs sr-create]] --port=&lt;port&gt; --destination-ip-address=&lt;network&gt; --destination-netmask=&lt;mask&gt; --gateway-ip-address=&lt;gateway&gt;</code>:
 
<pre style="font-size: smaller">
qs sr-create --port=ens192 --destination-ip-address=192.168.0.0 \
            --destination-netmask=255.255.0.0 --gateway-ip-address=10.0.8.1
qs sr-list
</pre>
 
=== Deleting a static route ===
 
{{Navigation|Storage Management &rarr; Storage Systems &rarr; Network Static Routes ''(tab)'' &rarr; ''select a route'' &rarr; Delete Static Route... ''(select + right-click)''}}
 
Select the '''Network Static Routes''' tab in the centre of the screen, right-click the route, and choose '''Delete Static Route...'''. Alternatively, expand the port in the '''Storage Systems''' section on the left, then right-click the route.
 
Routes can be created and deleted at any time. Deleting one withdraws it immediately. If a port is disabled or taken offline, the routes associated with it are disabled with it.
 
The CLI equivalent is <code>[[QuantaStor CLI Command Reference#static-route-delete|qs sr-delete]] --route-id=&lt;uuid&gt;</code>, where the UUID comes from <code>[[QuantaStor CLI Command Reference#static-route-list|qs sr-list]]</code>.
 
== Choosing which ports carry which traffic ==
 
QuantaStor works with all major 10/25/40/56/100GbE network interface cards (NICs) from Intel, Mellanox/NVIDIA, Broadcom, Cisco, Dell, HPE and Supermicro. Most systems have a mix of speeds, and the configuration that matters is which port carries which kind of traffic.
 
Clear '''iSCSI Portal''' and '''NVMeoF Portal''' on the slower 1GbE ports. That stops initiators from discovering and logging in through them, and reserves those ports for management traffic. A 1GbE port that answers an iSCSI discovery is a port some client will eventually run its I/O over.
 
Put the 1GbE management ports on a separate network from the ports serving NAS and SAN traffic. If management and storage share a network, the system can choose the 1GbE path for storage traffic and the resulting performance problem is hard to see from either end. Separate networks also keep [[#ARP Flux Protection|ARP flux protection]] effective, because each address is then reachable through exactly one port.
 
Configure the gateway on the management port only. Use the '''Firewall''' tab to block the protocols each port is not meant to serve, and the table in [[#TCP and UDP ports QuantaStor uses|TCP and UDP ports QuantaStor uses]] to see which TCP and UDP ports each service uses.
 
== TCP and UDP ports QuantaStor uses ==
 
Most organizations close every port a product does not need, to keep the attack surface small. This section lists every port a QuantaStor appliance listens on, what it is for, and which entry on the per-port [[#Firewall settings per port|Firewall]] tab controls it, so you can tell what can be closed and what has to stay open for the system to run normally. A dash in the Firewall tab column means the per-port firewall has no entry for that port; control it on your network firewall instead.
 
QuantaStor does not drop traffic by default. Its own iptables rules accept the ports it needs and drop only what you block on the Firewall tab, so on an appliance you have not locked down, every service below is reachable on every port that has an address.
 
=== Management and grid ===
 
{| class="wikitable"
! Port !! Protocol !! Service !! Firewall tab entry !! Notes
|-
| 443 || TCP || Web management interface (HTTPS) || -- || The address you browse to. Keep it open from the networks your administrators work from.
|-
| 80 || TCP || Web management interface (HTTP) || QS Web Management || Redirects to HTTPS on 443 by default. This is safe to close as you'll always be using HTTPS but the redirect is there for convenience.
|-
| 8080 || TCP || Web management interface (HTTPS, alternate port) || QS Web Management || Serves the same interface over HTTPS on a second port. This is available as a backup in case a service or firewall is setup to block it.  On example where this can be handy is if the object storage gateways is set to bind to 443 rather than 7480.  Safe to close it unless something depends on it.
|-
| 8153 || TCP || [[QuantaStor REST API Reference Guide|REST API]] (HTTPS) || QS REST API || Needed by scripts, automation, plugins and integrations that call the REST API. Safe to close if you're not using the QuantaStor REST API in scripts, QuantaStor Proxmox plug-in or the QuantaStor Python Client library.  The 'qs' CLI does not use this interface, it goes direct into the service API on 5151.
|-
| 5151 || TCP || QuantaStor core service (TLS) || -- || Every appliance in a [[Grid Configuration|storage grid]] talks to the others on this port, so it must be open between all grid members in both directions. The <code>qs</code> CLI also uses it when pointed at a remote system.
|-
| 22 || TCP || SSH || -- || Console access for administrators and OSNEXUS support, and the transport for encrypted remote replication (see below). Restrict it to administrator networks and to the other appliances you replicate with.
|}
 
The core service also listens on TCP 5152, bound to the loopback address only. It needs no rule on your network firewall.
 
=== Storage protocols ===
 
{| class="wikitable"
! Port !! Protocol !! Service !! Firewall tab entry !! Notes
|-
| 860, 3260 || TCP || iSCSI target || iSCSI Target || Initiator access to [[Storage Volumes]]. Clearing '''iSCSI Portal''' on a port also stops initiators logging in through it.
|-
| 4420 || TCP || NVMe-oF/TCP target || NVMeoF Target || Host access to [[Storage Volumes]] over [[NVMe-oF Target Configuration|NVMe-oF]]. Clearing '''NVMeoF Portal''' removes the listener on that address.
|-
| 2049, 111 || TCP and UDP || NFSv3 and NFSv4 || NFS || Client access to [[Network Shares]] over NFS. Port 111 is the RPC portmapper.
|-
| 2249 || TCP || NFS Ganesha || NFS Ganesha || The alternate NFS port used for scale-out (Ceph) file shares.
|-
| 137-139, 389, 445, 901 || TCP || SMB || SMB || Client access to [[Network Shares]] over SMB.
|}
 
=== Scale-out, clustering and replication ===
 
{| class="wikitable"
! Port !! Protocol !! Service !! Firewall tab entry !! Notes
|-
| 3300, 6789 || TCP || Ceph monitors || Ceph (6789 only) || Between the members of a [[Scale-out Block Setup (ceph)|Ceph cluster]], and from Ceph clients. The Ceph entry on the Firewall tab does not include 3300.
|-
| 6800-7100 || TCP || Ceph OSD and manager daemons || Ceph || Between the members of a Ceph cluster.
|-
| 7480, 7481 || TCP || Ceph object gateway (S3) || Ceph (7480 only) || S3 access to [[Ceph Object Storage|object storage]] buckets. A second gateway instance listens on 7481.
|-
| 5303-5332, 5403-5412 || TCP and UDP || Cluster heartbeat (Corosync) || -- || Between the appliances of an [[Setup Guide for Clustered HA Storage Pools|HA cluster]] or site cluster. QuantaStor also accepts multicast, IGMP and ICMP for the heartbeat. Blocking any of these splits the cluster.
|-
| 22 || TCP || Remote replication, encrypted || -- || The default. The source appliance sends the replication stream over SSH to the target appliance.
|-
| 20000-24999 || TCP || Remote replication, unencrypted || -- || Only for a replication link whose '''Encryption''' box is cleared. The target opens a receiver on a random port in this range for each transfer, and the source still uses SSH on 22 to start it. Open this range from source to target if you use unencrypted links.
|}
 
For [[Remote-replication (DR)|remote replication]] both appliances must be in the same grid, so TCP 5151 between them is needed as well as the replication transport.
 
=== Monitoring ===
 
{| class="wikitable"
! Port !! Protocol !! Service !! Firewall tab entry !! Notes
|-
| 3000 || TCP || [[Grafana Integration|Grafana]] || Grafana || Dashboards.
|-
| 9090, 9093, 9100, 9283 || TCP || [[Grafana Ceph Dashboard & Prometheus Integration|Prometheus]] || Prometheus || Prometheus server, Alertmanager, node exporter and the Ceph manager exporter.
|-
| 8888 || TCP || [[Chronograf Integration|Chronograf]] || Chronograf || Real-time statistics dashboard.
|-
| 161 || UDP || SNMP agent || -- || Only when the [[SNMP Agent Setup|SNMP agent]] is in use.
|}
 
The ports behind each Firewall tab entry are read from {{Code|1=/opt/osnexus/quantastor/conf/qs_services_firewall.conf}} on the appliance, and the static accept rules from the QuantaStor iptables service. Check those files on your own system if the table above and your appliance ever disagree.
 
== External sites QuantaStor connects to ==
 
These are the outbound connections an appliance makes. If your firewall restricts outbound traffic, allow the ones for the features you use. Every one of them is optional for serving storage: an appliance with no outbound access at all still serves its pools, shares and volumes, but cannot be activated online, upgraded from the public repositories or send logs to support.
 
{| class="wikitable"
! Purpose !! Destination !! Port !! Notes
|-
| Online license activation || <code>licensing.osnexus.com</code> || TCP 443 (HTTPS) || Used when you activate a license online and when the system reports license status. Without it, activate offline instead -- see [[Storage System Activate License Offline]] and [[License Management]].
|-
| Upgrades (QuantaStor packages) || <code>packages.osnexus.com</code> || TCP 80 (HTTP) || The QuantaStor package repository the [[Upgrade Manager]] installs from.
|-
| Upgrades and [[Security Updates|security updates]] (operating system) || <code>archive.ubuntu.com</code>, <code>us.archive.ubuntu.com</code>, <code>security.ubuntu.com</code> || TCP 80 (HTTP) || The Ubuntu package mirrors for the base operating system. Where appliances have no Internet access, use a [[QuantaStor Local Update Mirror|local update mirror]] instead.
|-
| [[Send Support Log Files|Sending support logs]] || <code>jira.osnexus.com</code> || TCP 21 (FTP, passive) || The first upload attempt. The log bundle is encrypted before it leaves the system.
|-
| Sending support logs (fallback) || <code>services.osnexus.com</code> || TCP 443 (HTTPS) || Used when FTP is unreachable. When the system has an HTTP proxy configured, QuantaStor skips FTP and uploads over HTTPS through the proxy.
|-
| Sending support logs (collection list) || <code>qstor-downloads.s3.amazonaws.com</code> || TCP 80 (HTTP) || Fetches the latest list of what to collect before building the bundle.
|-
| Email alerts || Your SMTP server || The port you configure || Set in the [[Storage System Alert Manager|Alert Manager]]. See [[Call-home / Alerting]].
|-
| Alert integrations || The service's own endpoint, for example [[Slack Integration|Slack]], [[PagerDuty Integration|PagerDuty]] or [[OpsGenie Integration|OpsGenie]] || TCP 443 (HTTPS) || Only for the integrations you configure.
|-
| Time synchronization || Your NTP servers, or <code>ntp.ubuntu.com</code> when none are set || UDP 123 || Set NTP servers per system or for the whole grid -- see [[#Where network ports are managed|Where network ports are managed]].
|-
| Name resolution || Your DNS servers || UDP and TCP 53 || Every external site on this list is reached by name.
|-
| Encryption key server || Your KMIP key server || TCP 5696 by default || Only when a [[Create Key Server Profile|key server profile]] is configured.
|-
| Cloud backup and cloud containers || The cloud provider's endpoint, such as <code>s3.amazonaws.com</code> || TCP 443 (HTTPS) || Only for the [[Cloud Containers / NAS Gateway|cloud containers]] and [[Backup Policies|backup policies]] you configure.
|-
| Remote replication || The other appliance's address || TCP 22, TCP 5151, and TCP 20000-24999 for unencrypted links || See [[#Scale-out, clustering and replication|Scale-out, clustering and replication]] above.
|}
 
== Locking down the network ==
 
Use two layers. The per-port [[#Firewall settings per port|Firewall]] tab and [[Firewall Management]] block services by destination address on the appliance itself; your network firewall handles everything those settings have no entry for, and all outbound traffic.
 
* '''Keep open between grid members:''' TCP 5151. Add TCP 22 between systems that replicate to each other, the Corosync ranges between HA and site cluster members, and the Ceph ports between scale-out cluster members.
* '''Keep open from administrator networks:''' TCP 443 for the web interface, TCP 22 if you use SSH, and TCP 8153 if anything calls the REST API.
* '''Keep open from clients:''' only the storage protocols they use -- iSCSI, NVMe-oF, NFS, SMB or S3 -- and only on the ports that serve them.
* '''Safe to close if unused:''' TCP 80 and 8080, the monitoring ports, NFS Ganesha, and every storage protocol you do not serve.
 
Block per port rather than system-wide where you can. A storage port that only serves iSCSI should block NFS, SMB and the web interface; a management port should block the storage protocols. After locking down, run the [[#Verifying connectivity|Network Check]] and send a test alert to confirm nothing you rely on was cut off.
 
== Verifying connectivity ==
 
{{Navigation|Storage Management &rarr; Storage Systems &rarr; ''select a Storage System'' &rarr; Health Checker ''(toolbar)''}}
 
The Network Check report tests reachability from each of the system's ports, including whether jumbo frames get through end to end, and reports the round-trip time for each destination. Run it after changing an MTU, since an MTU mismatch is exactly the kind of fault that leaves a port looking healthy while large transfers stall. See [[Network Check View]].
 
The CLI equivalent is <code>[[QuantaStor CLI Command Reference#network-check|qs net-check]]</code>:
 
<pre style="font-size: smaller">
qs net-check
</pre>
 
== Command line reference ==
 
Every operation on this page has a CLI equivalent. Add <code>--verbose</code> to any command to see its full argument list.
 
{| class="wikitable"
! Command !! Purpose
|-
| <code>[[QuantaStor CLI Command Reference#network-port-list|qs np-list]]</code> / <code>[[QuantaStor CLI Command Reference#network-port-get|qs np-get]]</code> || List all ports, or show one in detail.
|-
| <code>[[QuantaStor CLI Command Reference#network-port-modify|qs np-modify]]</code> || Change addressing, MTU, portal flags, autotuning, lossless networking and the per-port firewall.
|-
| <code>[[QuantaStor CLI Command Reference#network-port-enable|qs np-enable]]</code> / <code>[[QuantaStor CLI Command Reference#network-port-disable|qs np-disable]]</code> || Bring a port online or offline.
|-
| <code>[[QuantaStor CLI Command Reference#network-port-restart|qs np-restart]]</code> / <code>[[QuantaStor CLI Command Reference#network-port-rescan|qs np-rescan]]</code> || Restart one port, or rediscover all ports.
|-
| <code>[[QuantaStor CLI Command Reference#network-port-rename|qs np-rename]]</code> / <code>[[QuantaStor CLI Command Reference#network-port-hide|qs np-hide]]</code> || Rename a port, or hide it from the interface.
|-
| <code>[[QuantaStor CLI Command Reference#bonded-interface-create|qs bond-create]]</code> / <code>[[QuantaStor CLI Command Reference#bonded-interface-delete|qs bond-delete]]</code> || Create and delete bonded ports.
|-
| <code>[[QuantaStor CLI Command Reference#vlan-interface-create|qs vlan-create]]</code> / <code>[[QuantaStor CLI Command Reference#vlan-interface-delete|qs vlan-delete]]</code> || Create and delete VLAN interfaces.
|-
| <code>[[QuantaStor CLI Command Reference#virtual-interface-create|qs vif-create]]</code> / <code>[[QuantaStor CLI Command Reference#virtual-interface-delete|qs vif-delete]]</code> || Create and delete virtual interfaces.
|-
| <code>[[QuantaStor CLI Command Reference#static-route-create|qs sr-create]]</code> / <code>[[QuantaStor CLI Command Reference#static-route-delete|qs sr-delete]]</code> / <code>[[QuantaStor CLI Command Reference#static-route-list|qs sr-list]]</code> / <code>[[QuantaStor CLI Command Reference#static-route-get|qs sr-get]]</code> || Manage static routes.
|-
| <code>[[QuantaStor CLI Command Reference#ha-interface-create|qs hai-create]]</code> / <code>[[QuantaStor CLI Command Reference#site-vif-create|qs svr-create]]</code> || Create HA pool and site cluster virtual interfaces.
|-
| <code>[[QuantaStor CLI Command Reference#network-check|qs net-check]]</code> || Run the network reachability and jumbo frame report.
|}
 
Note that the <code>--port-type</code> argument of <code>[[QuantaStor CLI Command Reference#network-port-modify|qs np-modify]]</code> documents only <code>static</code> and <code>dhcp</code>. To remove a port's configuration entirely, use the '''Config Type''' setting of <code>disabled</code> in the web interface.
 
Set the ARP Filtering policy in the web interface, as described in [[#ARP Filtering policy|ARP Filtering policy]].
 
== Related pages ==
 
* [[Storage System#DNS Servers|Storage System]] -- the DNS, NTP and network settings on the Storage System Modify dialog
* [[Storage System Modify]] -- the Network Settings tab, which holds the ARP Filtering policy
* [[Network Port Usage]] -- the older stand-alone list of TCP and UDP ports
* [[Firewall Management]] -- system-level firewall settings, which the per-port settings inherit from
* [[Storage System]] -- the storage system object these ports belong to
* [[Storage Pools]] -- pools whose HA failover groups own the floating virtual interfaces
* [[Network Shares]] -- NAS shares served over the ports configured here
* [[Storage Volumes]] -- SAN volumes served over the iSCSI and NVMe-oF portals configured here
* [[Setup Guide for Clustered HA Storage Pools]] -- cluster heartbeat and HA virtual interface setup
* [[Remote-replication (DR)]] -- replication between appliances
* [[QuantaStor Local Update Mirror]] -- upgrading appliances that have no Internet access
* [[High-availability VIF Management]] -- how floating cluster addresses fail over
* [[QuantaStor Touch Files]] -- touch files, including the one that turns ARP flux protection off
* [[Network Check View]] -- the network reachability report
* [[QuantaStor CLI Command Reference]] -- full argument lists for every command named on this page
 
----
<small>''Verified against QuantaStor 6.9.0.''</small>

Latest revision as of 05:45, 7 October 2026


Network ports (also called target ports or network interfaces) are the Ethernet interfaces through which a QuantaStor system is managed and through which it serves storage. This page covers how to address a port, how to combine ports into a bond, how to layer VLAN and virtual interfaces on top of a port, how to control which protocols each port carries, how QuantaStor keeps ARP answers on the right port, how to add static routes, and which TCP and UDP ports and external sites QuantaStor needs so you can lock the network down without breaking it.

Client hosts reach Storage Volumes over iSCSI and NVMe-oF/TCP through a network port, authorized users reach Network Shares over NFS and SMB through a network port, and object storage buckets are reached over S3 through a network port. Every one of those paths depends on the port configuration described here, so it is worth getting right before you provision storage.

Section Purpose
Port types and naming How to tell a physical port, a bond, a VLAN, a VIF and a cluster VIF apart by name.
Where network ports are managed The tabs, toolbar and right-click menu that hold every port operation.
Static vs DHCP assigned IP addresses Why static addressing is required in practice, and the one-time conversion on a fresh install.
Modifying network port settings Every field on the General Settings tab of Modify Network Port.
Firewall settings per port Allowing and blocking individual protocols on one port.
NIC bonding and trunking Combining ports for throughput and fault tolerance, and which bonding mode to pick.
VLAN interfaces Tagged virtual ports on a physical or bonded port.
Virtual interfaces Extra IP addresses on one port.
High-availability and cluster virtual interfaces Floating IPs that move with a pool, a site or the grid.
ARP Flux Protection How QuantaStor stops a multi-homed system answering ARP on the wrong port, and the ARP Filtering policy.
Renaming a network port Giving a port a stable, meaningful name.
Online, offline, restart and rescan Bringing a port up or down and picking up out-of-band changes.
Static route management Routing specific networks through a gateway other than the default.
Choosing which ports carry which traffic Separating management traffic from storage traffic.
TCP and UDP ports QuantaStor uses Every port the appliance listens on, why, and which ones the Firewall tab can block.
External sites QuantaStor connects to Outbound connections for licensing, upgrades, support logs, alerting and replication.
Locking down the network What you can safely block, and what has to stay open.
Verifying connectivity The Network Check report.
Command line reference The qs commands that cover everything on this page.

Port types and naming

QuantaStor names every port after the interface the kernel presents, and uses two delimiters to indicate that a port is layered on top of another one. Reading the name tells you what a port is:

Name Type Notes
ens192, eno1, enp3s0f0 Physical port The predictable name assigned by the kernel. Rename it if you want something more meaningful -- see Renaming a network port.
bond0 Bonded port Two or more physical ports combined. The member ports become slaves and stop carrying their own IP address.
ens192.56 VLAN interface A period separates the parent port name from the VLAN tag. A VLAN on a bond looks like bond0.56.
ens192:1 Virtual interface (VIF) A colon followed by an index, allocated from 1 upwards. An extra IP address on the parent port.
ens192:ha345680 HA pool virtual interface Floats with a storage pool when the pool fails over.
ens192:sv... Site cluster virtual interface Floats between the appliances of a site cluster.
ens192:gm Grid management virtual interface The single floating IP a storage grid is managed through. iSCSI and NVMe-oF are permanently disabled on it, because a management IP that moves on failover is not a safe place to export block storage from.
ens192 (S) Manually configured secondary port An address that QuantaStor discovered but does not own. Management operations are refused on these.

Interface names are limited to 15 characters by the kernel, and the tag QuantaStor appends when it creates a cluster virtual interface has to fit inside that budget. A long parent port name is therefore not merely untidy -- it will eventually block cluster VIF creation, so keep renamed ports short.

Where network ports are managed

The Network Ports tab lists every port on every system in the grid, grouped by storage system. The Network Port toolbar group above it holds the create, delete and state operations.
Navigation: Storage Management → Storage Systems → select a Storage System → Network Ports (tab)

Select the Storage Systems section in the tree on the left, then the Network Ports tab in the centre of the screen. The tab lists every port on every system in the grid, grouped by storage system, with its state, link state, link speed, IP address, subnet mask, whether it is an iSCSI or NVMe-oF portal, and its vendor, model, MAC address, byte counters and MTU. The Network Static Routes tab beside it lists the static routes on those same ports.

The Network Port toolbar group at the top of the screen carries Modify, Create Virtual Port, Create VLAN Port, Create Bonded Port, the matching Delete operations, Online, Offline and Rescan All.

Right-clicking a port in the Network Ports tab reaches every port operation, including the three that are not on the toolbar: Rename, Restart, and Create/Delete Static Route.

Three operations are reachable only by right-clicking a port, not from the toolbar: Rename Network Port..., Restart Network Port... and Create Static Route... / Delete Static Route.... Right-click a port either in the tree on the left or in the Network Ports tab.

For grid-wide settings such as the DNS servers and search domain that apply to all ports, use Modify Grid Network Settings instead -- the Modify Grid button in the Storage System Grid toolbar group. It writes one DNS search suffix, DNS server list and NTP server list to every system you tick, overwriting whatever those systems had. The per-system equivalents are the DNS Servers and NTP Servers tabs of Storage System Modify.

Static vs DHCP assigned IP addresses

QuantaStor supports both statically assigned IP addresses and dynamically assigned Dynamic Host Configuration Protocol (DHCP) addresses. After installation the first port is typically configured by DHCP and the rest are disabled, so that the appliance is reachable for initial setup.

Reconfigure every port with a static IP address before you provision storage. A DHCP lease can change, and a storage system whose address moves takes its iSCSI targets, NFS exports and SMB shares with it. The appliance also has to be able to boot and serve storage when the DHCP server is unreachable. The only defensible use of DHCP is a reservation that always hands the same address to the same MAC, and even then a static address is simpler to reason about.

There is a second, one-time reason to set a static management address first. A freshly installed system still has the installer's netplan configuration active. The first time you modify any network port, QuantaStor converts that configuration into /etc/network/interfaces, backs up the previous file as /etc/network/interfaces-netplan-convert-<timestamp>.backup, renames each /etc/netplan/*.yaml file to .bak, and disables the netplan renderer. All interfaces are torn down and brought back during that conversion, so a port still on DHCP can come back with a different address and the management session drops. QuantaStor guards against this: while any port is in that state it is flagged with a warning whose detail begins "Netplan", and the Modify Network Port dialog refuses any change other than to static, with the message "Be sure to configure a static management network port before modifying network ports."

Set a static address on your management port first, then work through the remaining DHCP ports until they all have static IPs assigned. Reboot afterwards and confirm the configuration, which is what the accompanying alert asks for. This is an important step after an initial QuantaStor OS installation as this allows the system to cleanup network configuration issues left behind by the text mode installer.

Modifying network port settings

The General Settings tab of Modify Network Port. The Static IP Configuration Settings fieldset is only enabled when Config Type is set to static.
Navigation: Storage Management → Storage Systems → Network Port (select + right-click) → Modify Network Port...

The dialog is also on the Modify button in the Network Port toolbar group. It has two tabs, General Settings and Firewall.

General Settings

Field Description
Storage System The system whose port you are configuring. Changing it reloads the Network Port list.
Network Port The port to modify. Every port on the selected system is offered, including bonds, VLANs and virtual interfaces.
Config Type static, dhcp or disabled. Only static enables the Static IP Configuration Settings fieldset below. Choosing disabled prompts for confirmation and then removes the port's stanza from /etc/network/interfaces entirely, so the port stops carrying traffic and does not come back after a reboot. A port that is a bond slave reports as disabled and cannot be changed here -- delete the bond first. Cluster virtual interfaces are locked to static because the cluster software owns their addresses.
Description An optional note recorded against the port. Use it to record what the port is for; there is no other place to do that.
iSCSI Portal Makes this port's IP address an allowed iSCSI portal. Clearing it removes the address from the allowed-portal list of every iSCSI target on the system, so initiators can no longer log in through it. Existing sessions are not torn down -- the change takes effect on the next login. If no port at all is left iSCSI-enabled, QuantaStor restricts iSCSI to loopback rather than leaving it open.
NVMeoF Portal Makes this port's IP address an NVMe-oF/TCP listener on port 4420. Unlike the iSCSI setting, clearing it removes the listener, which does drop connections through that address.
Lossless Networking Configures RoCEv2 with priority flow control on the port. Only selectable on adapters that support it -- in practice Mellanox/NVIDIA ConnectX cards with RDMA enabled -- and greyed out everywhere else. It sets DSCP-based priority trust, enables PFC on priority 3 only, turns global pause off, sizes the receive buffers for the adapter model, and records the port in /var/opt/osnexus/quantastor/conf/qs_losslessNetworking.conf. QuantaStor verifies the result and reverts the change if verification fails. Enable it only where the switch fabric is configured for the same priority flow control scheme; a one-sided configuration performs worse than plain Ethernet.
Optimize hardware RX/TX buffer settings for throughput Enlarges the adapter's receive and transmit ring buffers. QuantaStor records the current sizes so it can put them back, then raises both rings to the largest power-of-two multiple of 128 that still fits under the adapter's hardware maximum -- roughly half the maximum, which improves large-block and replication throughput without asking the driver for a size it will reject. Settings are recorded in /var/opt/osnexus/quantastor/conf/qs_porttunings.conf and re-applied every five minutes. Available on physical ports only: a bond, VLAN or virtual interface has no hardware ring of its own, so enable it on the member ports instead.

The Static IP Configuration Settings fieldset holds the addressing:

Field Description
IP Address The static address for the port. Each port needs a unique address; QuantaStor rejects an address already recorded on another port. If the address is part of a Ceph cluster configuration you have to disable the port before the address can be changed.
Subnet Mask The mask for the port's network, such as 255.255.0.0.
Gateway Optional. QuantaStor checks that the gateway is on the same subnet as the port address and refuses it otherwise. Configure a gateway on one port only. QuantaStor gives every static port the same route metric, so a second gateway-bearing port ends up competing for the default route rather than adding a fallback.
MTU and Jumbo Frames The maximum transmission unit, 1492 to 64000. The Jumbo Frames button toggles between 9000 and 1500 and relabels itself Default Frames once jumbo is set. Set 9000 on all 10GbE and faster interfaces. If you move away from 1500, the switch ports and the client hosts on that network have to match -- a mismatched MTU produces intermittent stalls rather than a clean failure. The field is disabled for VLAN and virtual interfaces, which inherit the parent port's MTU; changing the parent cascades the new MTU to all of its children. Setting a VLAN MTU above its parent's is capped to the parent's value with an alert.

Changing the IP address of a port that has static routes configured is flagged: the routes are bound to the old address and client connections through them can drop. Remove the routes first.

Click Apply to commit the change and keep the dialog open, or OK to commit and close.

Firewall settings per port

The Firewall tab sets each protocol to Allow, Block or Inherit for this port only.
Navigation: Storage Management → Storage Systems → Network Port (select + right-click) → Modify Network Port... → Firewall (tab)

The Firewall tab lists each service QuantaStor knows how to filter, with its description and a Status setting. The three settings are what makes this useful, and the distinction is easy to miss:

  • Inherit -- follow the system-wide firewall setting. This is the default for every service on every port.
  • Block -- drop traffic for this service arriving at this port's IP address, even though the system-wide setting allows it.
  • Allow -- accept traffic for this service at this port's IP address, even though the system-wide setting blocks it.

So the system level sets the policy and the port level carries the exceptions. The usual pattern is to block everything you do not use at the system level -- see Firewall Management -- and then Allow the handful of protocols each port is actually meant to serve.

Two things are worth knowing before you rely on it. The rules match on the port's destination IP address, not on the interface, so a per-port rule really follows the address; that is deliberate, and it is why the masks are also recorded against HA and site virtual interfaces so the rule follows a floating address to whichever system holds it. And because QuantaStor reports a service as blocked whenever any DROP rule exists for it, blocking a protocol on one port raises an alert saying that service is blocked system-wide, even though only that one address is affected.

The service list, the TCP and UDP ports behind each entry, and the bit each one occupies come from /opt/osnexus/quantastor/conf/qs_services_firewall.conf on the appliance. The rules themselves are iptables rules in QuantaStor's own chains (QUANTASTOR-INPUT, QUANTASTOR-FORWARD, QUANTASTOR-OUTPUT), written out to /var/opt/osnexus/quantastor/iptables.qsfirewallrules so they survive a restart, and kept out of the shared chains so the firewall can be rebuilt without disturbing other rules on the system. QuantaStor re-applies them on change and again every 20 minutes.

Changing a setting that makes a service unreachable prompts for confirmation first. For the TCP and UDP ports behind each entry, and the ports the Firewall tab does not cover, see TCP and UDP ports QuantaStor uses below.

The same firewall settings can be driven from the CLI with qs network-port-modify --protocol-block= and --protocol-allow=, each taking a comma-separated protocol list, with --protocol-block-reset and --protocol-allow-reset to clear a list. Prefix a protocol with a tilde (~) to remove it from a list rather than add it.

NIC bonding and trunking

Create Bonded Port. At least two ports must be selected in the grid; clear Hide Configured Ports to bond a port that already has an IP address.
Navigation: Storage Management → Storage Systems → Network Port → Create Bonded Port (toolbar)

Bonding, also called trunking or teaming, combines several physical ports into one logical port to improve throughput and to survive the loss of a cable, a switch port or a NIC. The bond takes the IP address; the member ports become slaves and no longer carry addresses of their own.

For every mode except LACP, all of the member ports must be connected to the same switch. LACP works across switches -- which is what makes it the only mode that survives a whole switch going away -- but it requires the switch ports to be configured for LACP as well. QuantaStor surfaces switch-side mistakes on the port itself, reporting an invalid partner MAC address or a partner that is not advertising LACP support, so check the port state after creating an LACP bond rather than assuming it came up.

The dialog fields are the same addressing fields as Modify Network Port, plus:

Field Description
Bond Mode The bonding policy. See the table below.
Select the Network Ports to be Bonded Select at least two ports. The grid lists each candidate port's name, IP address, vendor and model.
Hide Configured Ports Selected by default, hiding ports that already have an IP address so you do not accidentally bond your management port. Clear it to bond a configured port; QuantaStor warns that the port's existing IP configuration will be removed and that the bond will only have the address you entered.
Force Proceeds without the confirmation prompt when a selected port already has an IP configuration.

The modes QuantaStor offers, with the switch requirement each one carries:

Mode Kernel mode Switch Behaviour
Link Aggr Ctrl Protocol (LACP layer2) 802.3ad, hash layer2 Managed, LACP configured Balances by source and destination MAC. Spans switches. Recommended default.
Link Aggr Ctrl Protocol (LACP layer2+3) 802.3ad, hash layer2+3 Managed, LACP configured Adds the IP addresses to the hash, which spreads traffic better when many clients sit behind one router MAC.
Link Aggr Ctrl Protocol (LACP layer3+4) 802.3ad, hash layer3+4 Managed, LACP configured Adds the TCP/UDP ports to the hash, so a single client with many connections can use more than one member link.
Round Robin (balance-rr) balance-rr Etherchannel, managed Sends packets in sequence across the members. Highest single-stream throughput, but it reorders packets and does not span switches.
Adaptive Transmit Load Balancing (balance-tlb) balance-tlb Unmanaged Balances outbound traffic only; inbound arrives on one member. Needs no switch configuration.
Adaptive Load Balancing (balance-alb) balance-alb Unmanaged As balance-tlb, plus inbound balancing via ARP negotiation. Needs no switch configuration.
Balance XOR (balance-xor) balance-xor Etherchannel, managed Picks a member by hashing the MAC addresses. Static, so both ends must agree.
Active-Backup (active-backup) active-backup Unmanaged One member carries all traffic; the others stand by. No throughput gain, and the simplest mode to get right.

For fault tolerance that survives a switch outage, LACP is the mode to use. Where the switches are unmanaged and cannot be configured, use Active-Backup or one of the adaptive load balancing modes.

The Bond Mode combo in the Create Bonded Port dialog is pre-selected from the storage system's own default bonding policy, which QuantaStor sets on first startup and records in /etc/modprobe.d/bonding.conf. On the 6.9.0 system this page was verified against that default was LACP layer 2. Check yours with qs system-get and look at the Bond Mode field, and change it with qs system-modify --bond-mode=<mode>. QuantaStor sets miimon=100 on every bond so link failures are detected within a tenth of a second, and allows a maximum of four bonds per system.

To change the mode of a bond that already exists, open Modify Network Port on the bond and use the Bond Port Advanced Settings... button, which appears only when the selected port is a bond. The mode cannot be changed while the bond is carrying a virtual interface or an HA virtual interface -- remove those first. If a member port drops out of a bond, QuantaStor re-enslaves it automatically, at most once an hour per member.

Delete a bond with Delete Bonded Port in the toolbar, or qs bond-delete --port=<bond>. The member ports return to being unconfigured physical ports.

VLAN interfaces

Create VLAN Interface. The MTU is read-only because a VLAN inherits it from the parent port.
Navigation: Storage Management → Storage Systems → Network Port → Create VLAN Port (toolbar)

A VLAN (Virtual LAN) interface is a virtual port that tags its traffic with a VLAN ID, letting one physical or bonded port serve several isolated networks. VLAN interfaces can be created and deleted at any time.

Field Description
Storage System The system to create the VLAN interface on.
Parent Port The physical or bonded port to tag on. A VLAN cannot be created on another VLAN, nor on a virtual interface, so those are filtered out of the list.
IP Address, Subnet Mask, Gateway The addressing for the VLAN interface, on the network that VLAN reaches.
Description An optional note. Recording which network the tag corresponds to pays for itself.
MTU Read-only. A VLAN inherits the parent port's MTU; change it on the parent.
VLAN ID The 802.1Q tag, 1 to 4094 (4095 is reserved). Defaults to 5.
VLAN QoS The 802.1p Class of Service user priority to record for the interface, 0 (lowest, the default) to 7 (highest).

The new port is named after its parent with a period and the tag, so VLAN 56 on ens192 becomes ens192.56 and VLAN 56 on bond0 becomes bond0.56. Delete a VLAN interface with Delete VLAN Port in the toolbar or qs vlan-delete --port=<port>. A port that has a VLAN child cannot be disabled or renamed until the VLAN is removed.

Virtual interfaces

Create Virtual Interface. A VIF adds a second address to an existing port.
Navigation: Storage Management → Storage Systems → Network Port → Create Virtual Port (toolbar)

A virtual interface (VIF) gives a physical or bonded port an additional IP address, so one port can be present on several networks or carry several addresses on the same network. VIFs can be created and deleted at any time, and cannot be layered on top of another VIF.

The fields are Storage System, Parent Port, IP Address, Description, Subnet Mask, Gateway, and a read-only MTU inherited from the parent port. QuantaStor names the new interface after the parent with a colon and the next free index, so the first VIF on ens192 is ens192:1.

Two constraints are not obvious from the dialog. A VIF cannot be created on a bond running in Active-Backup mode. And DHCP is not valid on a virtual interface -- a VIF is always statically addressed.

Delete a VIF with Delete Virtual Port in the toolbar or qs vif-delete --port=<port>.

High-availability and cluster virtual interfaces

QuantaStor also has virtual interfaces that float between systems. You will see these with :ha, :sv or :gm in the name, such as ens192:ha345680.

  • HA pool virtual interfaces (:ha) are associated with a specific storage pool and move with the pool when it fails over to another system. A pool can have several; each additional one increments the last digit of the port name, so they run :ha345680, :ha345681, :ha345682 and so on. Create them by right-clicking an HA failover group in the Storage Pools section and choosing Add Cluster VIF..., or with qs hai-create --ha-group=<group> --parent-port=<port> --ip-address=<ip>. A cluster heartbeat must be configured between the systems that manage the pool before an HA VIF can be created; see Setup Guide for Clustered HA Storage Pools and Storage Pool HA Failover Interface Create.
  • Site cluster virtual interfaces (:sv) float between the appliances of a site so that a service address stays available. See High-availability VIF Management.
  • The grid management virtual interface (:gm) is the single floating address a storage grid is managed through. There is exactly one per site cluster, and iSCSI and NVMe-oF are permanently disabled on it.

These are all managed from the High-availability VIF Management main tab, and by the cluster software rather than by the network port configuration -- that page covers use cases, failover, deliberate moves and location constraints. That is why the Modify Network Port dialog locks their Config Type to static, disables the addressing fields, and refuses IP, MTU and bond mode changes -- and why static routes cannot be created on them. Change a floating address through the HA or site VIF dialogs, not through Modify Network Port.

A floating address only works if the switches believe the system that currently holds it, which is what ARP flux protection, below, is for.

ARP Flux Protection

A system with more than one network port -- every HA node and every Ceph node, and any system with a separate management and storage network -- is multi-homed. With the Linux kernel's default ARP settings, any port on a multi-homed system will answer an ARP request for any address configured anywhere on the system, and the system may use any of its own addresses as the source of the ARP requests it sends. The switches then learn the wrong MAC address and switch port for an address, and traffic meant for the storage network arrives on the management port, or the other way round. This is known as ARP flux.

ARP flux matters most for the floating addresses described above. When an HA or cluster virtual interface moves, the system taking it over sends a gratuitous ARP so that the switches and clients update their tables. If the switches have already learned the wrong port for that address, or if another port answers for it, clients keep sending to the old location and the failover looks like an outage. On a bond carrying HA virtual interfaces the effect is worse, because the address also has to be learned through the right member link.

QuantaStor guards against this with two separate mechanisms. One is always on; the other is the ARP Filtering policy in Storage System Modify.

Always-on ARP flux protection

When the QuantaStor service starts, it sets four kernel parameters:

Parameter Value Effect
net.ipv4.conf.all.arp_ignore 1 Answer an ARP request only when the requested address is configured on the port the request arrived on.
net.ipv4.conf.default.arp_ignore 1 The same, for ports created after the service started.
net.ipv4.conf.all.arp_announce 2 Always use the best local address for the destination as the source of an ARP request, never an address that belongs to another port.
net.ipv4.conf.default.arp_announce 2 The same, for ports created after the service started.

The kernel applies the larger of the all value and a port's own value, so the all settings cover every existing port and the default settings cover bonds, VLANs and virtual interfaces created later. QuantaStor applies the values immediately and also writes them to /etc/sysctl.d/70-quantastor-arp.conf, so they survive a reboot and an upgrade takes effect without one. The file is managed by QuantaStor; do not edit it, because QuantaStor rewrites it on the next service start.

There is no setting for this in the web interface. It needs none: with these values a port still answers every ARP request for its own addresses, so single-homed systems behave exactly as before.

To check the live values on an appliance:

sysctl net.ipv4.conf.all.arp_ignore net.ipv4.conf.all.arp_announce
cat /etc/sysctl.d/70-quantastor-arp.conf

Turning it off

Turn ARP flux protection off only if your network design depends on the kernel's default ARP behaviour, for example a host-based load balancing scheme that expects any port to answer for any address. Create the touch file and restart the QuantaStor service:

touch /var/opt/osnexus/quantastor/touchfiles/tf_arp_flux_protection.disable
systemctl restart quantastor

On the next start QuantaStor deletes /etc/sysctl.d/70-quantastor-arp.conf and sets all four parameters back to the kernel default of 0. It reverts them only if that file exists, so values you set by hand with no QuantaStor file present are left alone. To turn the protection back on, delete the touch file and restart the service again. The setting is per system: on an HA pair or a cluster, do the same on every member, or the members will answer ARP differently. See QuantaStor Touch Files for touch files in general.

Setting disable_arp_logic=true in the [system] section of /etc/quantastor.conf has the same effect, and also stops QuantaStor from managing the ARP Filtering policy below.

ARP Filtering policy

The ARP Filtering fieldset on the Network Settings tab of Storage System Modify. Status shows whether filtering is currently in effect, here Disabled under the Auto policy on a system with no bond or virtual port.
Navigation: Storage Management → Storage Systems → select a Storage System → Modify (toolbar) → Network Settings (tab)

The ARP Filtering fieldset on the Network Settings tab of Storage System Modify controls a second kernel parameter, arp_filter. With arp_filter on, the kernel answers an ARP request on a port only if it would route traffic for the requester out of that same port. It adds to the always-on protection when two ports sit on the same subnet, which is where arp_ignore alone cannot decide which port should answer.

Field Description
ARP Policy Auto (the default), Enabled or Disabled. Auto turns filtering on when the system has a bonded port or any other virtual port, and off otherwise. Enabled and Disabled force it on or off regardless of the ports.
Status Read-only. Whether ARP filtering is in effect right now, read from the running kernel: Enabled or Disabled.

QuantaStor writes the setting to /etc/sysctl.conf as net.ipv4.conf.all.arp_filter so it survives a reboot, applies it to the running kernel, and, if the value changed, flushes the system's ARP table so neighbours are relearned through the right ports.

Under Auto, QuantaStor re-evaluates the setting when the service starts, when you change the policy, and when a bond is deleted. Creating a bond turns filtering on immediately. Creating a VLAN or virtual interface does not, by itself, re-evaluate the policy; the change is picked up the next time the service starts. If you are building a multi-homed configuration and want filtering on from the first port, set the policy to Enabled.

Leave the policy on Auto unless you have a specific reason not to. Use Enabled on a system that has two or more physical ports on the same network without a bond. Use Disabled only if filtering interferes with a routing design that deliberately sends replies out of a different port from the one the request arrived on.

Renaming a network port

Rename Network Port. The Port Information fieldset identifies the port you are about to rename.
Navigation: Storage Management → Storage Systems → Network Port (select + right-click) → Rename Network Port...

Predictable kernel names such as enp3s0f1 say where the adapter is but nothing about what it does. Renaming a port gives it a name that survives hardware changes and reads clearly in the port list.

Enter the new name in New Name; the Port Information fieldset below shows the selected port's UUID, IP address, MAC address and state so you can confirm you have the right one. QuantaStor writes a systemd link file at /etc/systemd/network/10-<newname>.link that matches the port by MAC address, renames the running interface, and rewrites the port's stanza in /etc/network/interfaces.

The dialog's own guidance is worth following: prefix your management port with mgmt, and avoid the eth prefix, which collides with the names kernel drivers hand out. Interfaces are conventionally numbered from zero, as in eno0, eno1. Keep the name under 15 characters -- see Port types and naming. The name is lowercased for you.

A port cannot be renamed while it is a bond slave, while it has VLAN or virtual interface children, or while the system is using legacy ethN naming mode. Remove the bond or the children first, or switch back to predictable naming in Modify Storage System. A reboot may be needed for the rename to take effect everywhere, and QuantaStor raises an alert saying so.

The CLI equivalent is qs np-rename --port=<port> --name=<newname>.

Online, offline, restart and rescan

Four state operations sit alongside the configuration dialogs:

Operation Where What it does
Online Network Port toolbar; right-click → Online Network Port... Brings the port up. The CLI equivalent is qs np-enable --port=<port>.
Offline Network Port toolbar; right-click → Offline Network Port... Takes the port down without discarding its configuration, so Online brings it back as it was. Static routes on the port go down with it and come back when it is brought back up. The CLI equivalent is qs np-disable --port=<port>.
Restart Network Port... Right-click only Cycles the port so a changed setting takes effect. The CLI equivalent is qs np-restart --port=<port>.
Rescan All Network Port toolbar; right-click → Rescan All Network Ports... Discovers newly added ports and picks up configuration made outside QuantaStor. The CLI equivalent is qs np-rescan.

Offlining a port that has virtual interfaces attached takes the virtual interfaces down with it. Setting Config Type to disabled in Modify Network Port is not the same thing as taking the port offline: offline is temporary, whereas disabled removes the configuration and does not survive a reboot.

Ports QuantaStor discovers but that you do not want cluttering the list can be hidden with qs np-hide --port=<port> --hide=true.

QuantaStor also repairs drift on its own. Discovery re-applies the recorded MTU, subnet mask and gateway when the live configuration diverges from the configuration file, at most once every six hours per correction, and marks the port with a warning telling you to use the modify operation rather than editing the system configuration directly.

Static route management

A static route customizes traffic flow so that a specific network is reached through a gateway other than the default. This is how you reach a network that the default gateway cannot route to, whether that is across a VLAN, through a VPN tunnel, or over a dedicated replication link. Each route has two parts: the gateway on the local network through which the remote network is reachable, and the address and mask of the remote network itself.

QuantaStor's static routes are per port, which gives granular control over which interface carries traffic for which network.

Creating a static route

Create Static Route. The gateway must be on the port's own network; the destination must be given in bind address form.
Navigation: Storage Management → Storage Systems → Network Port (select + right-click) → Create Static Route...

The dialog is also reachable by right-clicking inside the Network Static Routes tab.

Field Description
Storage System The system to add the route to.
Parent Port The port the route is bound to. Floating cluster interfaces are filtered out -- routes cannot be created on them.
Gateway IP Address The gateway on the port's own network through which the destination network is reached.
Destination IP Address The network to route to, in bind address form. QuantaStor checks the address against the mask and rejects anything that is not a network address, telling you the correct form to use.
Destination Subnet Mask The mask of the destination network.

For example, on a system whose ens192 port is 10.0.8.70/24 and which has a gateway at 10.0.8.1 that can reach the 192.168.0.0/16 network, enter a Gateway IP Address of 10.0.8.1, a Destination IP Address of 192.168.0.0 and a Destination Subnet Mask of 255.255.0.0. All traffic from that port destined for 192.168.0.0/16 is then sent via 10.0.8.1. Note that the gateway has to be on the port's own network -- QuantaStor applies the route immediately and refuses a gateway the port cannot reach.

Routes are activated the moment they are created, and QuantaStor refuses a gateway it cannot reach rather than recording a route that will never work. A duplicate destination network and mask on the same system is also refused.

The Network Static Routes tab lists each route with its parent port, gateway, and destination network and mask.

Route management is integrated with the platform rather than bolted on. Each route is written to an executable script at /etc/network/<port>.staticroutes, and QuantaStor installs hooks in /etc/network/if-up.d and /etc/network/if-down.d that run it. The routes therefore enable and disable correctly when a port is brought up or down with ifup and ifdown from an SSH session, not just from the web interface. QuantaStor treats those scripts as the source of truth and imports routes it finds in them, so a route added by hand is picked up rather than overwritten. We recommend using the qs CLI or the web management interface all the same, so the routes are recorded as objects you can see in the Network Static Routes tab.

The CLI equivalent is qs sr-create --port=<port> --destination-ip-address=<network> --destination-netmask=<mask> --gateway-ip-address=<gateway>:

qs sr-create --port=ens192 --destination-ip-address=192.168.0.0 \
             --destination-netmask=255.255.0.0 --gateway-ip-address=10.0.8.1
qs sr-list

Deleting a static route

Navigation: Storage Management → Storage Systems → Network Static Routes (tab) → select a route → Delete Static Route... (select + right-click)

Select the Network Static Routes tab in the centre of the screen, right-click the route, and choose Delete Static Route.... Alternatively, expand the port in the Storage Systems section on the left, then right-click the route.

Routes can be created and deleted at any time. Deleting one withdraws it immediately. If a port is disabled or taken offline, the routes associated with it are disabled with it.

The CLI equivalent is qs sr-delete --route-id=<uuid>, where the UUID comes from qs sr-list.

Choosing which ports carry which traffic

QuantaStor works with all major 10/25/40/56/100GbE network interface cards (NICs) from Intel, Mellanox/NVIDIA, Broadcom, Cisco, Dell, HPE and Supermicro. Most systems have a mix of speeds, and the configuration that matters is which port carries which kind of traffic.

Clear iSCSI Portal and NVMeoF Portal on the slower 1GbE ports. That stops initiators from discovering and logging in through them, and reserves those ports for management traffic. A 1GbE port that answers an iSCSI discovery is a port some client will eventually run its I/O over.

Put the 1GbE management ports on a separate network from the ports serving NAS and SAN traffic. If management and storage share a network, the system can choose the 1GbE path for storage traffic and the resulting performance problem is hard to see from either end. Separate networks also keep ARP flux protection effective, because each address is then reachable through exactly one port.

Configure the gateway on the management port only. Use the Firewall tab to block the protocols each port is not meant to serve, and the table in TCP and UDP ports QuantaStor uses to see which TCP and UDP ports each service uses.

TCP and UDP ports QuantaStor uses

Most organizations close every port a product does not need, to keep the attack surface small. This section lists every port a QuantaStor appliance listens on, what it is for, and which entry on the per-port Firewall tab controls it, so you can tell what can be closed and what has to stay open for the system to run normally. A dash in the Firewall tab column means the per-port firewall has no entry for that port; control it on your network firewall instead.

QuantaStor does not drop traffic by default. Its own iptables rules accept the ports it needs and drop only what you block on the Firewall tab, so on an appliance you have not locked down, every service below is reachable on every port that has an address.

Management and grid

Port Protocol Service Firewall tab entry Notes
443 TCP Web management interface (HTTPS) -- The address you browse to. Keep it open from the networks your administrators work from.
80 TCP Web management interface (HTTP) QS Web Management Redirects to HTTPS on 443 by default. This is safe to close as you'll always be using HTTPS but the redirect is there for convenience.
8080 TCP Web management interface (HTTPS, alternate port) QS Web Management Serves the same interface over HTTPS on a second port. This is available as a backup in case a service or firewall is setup to block it. On example where this can be handy is if the object storage gateways is set to bind to 443 rather than 7480. Safe to close it unless something depends on it.
8153 TCP REST API (HTTPS) QS REST API Needed by scripts, automation, plugins and integrations that call the REST API. Safe to close if you're not using the QuantaStor REST API in scripts, QuantaStor Proxmox plug-in or the QuantaStor Python Client library. The 'qs' CLI does not use this interface, it goes direct into the service API on 5151.
5151 TCP QuantaStor core service (TLS) -- Every appliance in a storage grid talks to the others on this port, so it must be open between all grid members in both directions. The qs CLI also uses it when pointed at a remote system.
22 TCP SSH -- Console access for administrators and OSNEXUS support, and the transport for encrypted remote replication (see below). Restrict it to administrator networks and to the other appliances you replicate with.

The core service also listens on TCP 5152, bound to the loopback address only. It needs no rule on your network firewall.

Storage protocols

Port Protocol Service Firewall tab entry Notes
860, 3260 TCP iSCSI target iSCSI Target Initiator access to Storage Volumes. Clearing iSCSI Portal on a port also stops initiators logging in through it.
4420 TCP NVMe-oF/TCP target NVMeoF Target Host access to Storage Volumes over NVMe-oF. Clearing NVMeoF Portal removes the listener on that address.
2049, 111 TCP and UDP NFSv3 and NFSv4 NFS Client access to Network Shares over NFS. Port 111 is the RPC portmapper.
2249 TCP NFS Ganesha NFS Ganesha The alternate NFS port used for scale-out (Ceph) file shares.
137-139, 389, 445, 901 TCP SMB SMB Client access to Network Shares over SMB.

Scale-out, clustering and replication

Port Protocol Service Firewall tab entry Notes
3300, 6789 TCP Ceph monitors Ceph (6789 only) Between the members of a Ceph cluster, and from Ceph clients. The Ceph entry on the Firewall tab does not include 3300.
6800-7100 TCP Ceph OSD and manager daemons Ceph Between the members of a Ceph cluster.
7480, 7481 TCP Ceph object gateway (S3) Ceph (7480 only) S3 access to object storage buckets. A second gateway instance listens on 7481.
5303-5332, 5403-5412 TCP and UDP Cluster heartbeat (Corosync) -- Between the appliances of an HA cluster or site cluster. QuantaStor also accepts multicast, IGMP and ICMP for the heartbeat. Blocking any of these splits the cluster.
22 TCP Remote replication, encrypted -- The default. The source appliance sends the replication stream over SSH to the target appliance.
20000-24999 TCP Remote replication, unencrypted -- Only for a replication link whose Encryption box is cleared. The target opens a receiver on a random port in this range for each transfer, and the source still uses SSH on 22 to start it. Open this range from source to target if you use unencrypted links.

For remote replication both appliances must be in the same grid, so TCP 5151 between them is needed as well as the replication transport.

Monitoring

Port Protocol Service Firewall tab entry Notes
3000 TCP Grafana Grafana Dashboards.
9090, 9093, 9100, 9283 TCP Prometheus Prometheus Prometheus server, Alertmanager, node exporter and the Ceph manager exporter.
8888 TCP Chronograf Chronograf Real-time statistics dashboard.
161 UDP SNMP agent -- Only when the SNMP agent is in use.

The ports behind each Firewall tab entry are read from /opt/osnexus/quantastor/conf/qs_services_firewall.conf on the appliance, and the static accept rules from the QuantaStor iptables service. Check those files on your own system if the table above and your appliance ever disagree.

External sites QuantaStor connects to

These are the outbound connections an appliance makes. If your firewall restricts outbound traffic, allow the ones for the features you use. Every one of them is optional for serving storage: an appliance with no outbound access at all still serves its pools, shares and volumes, but cannot be activated online, upgraded from the public repositories or send logs to support.

Purpose Destination Port Notes
Online license activation licensing.osnexus.com TCP 443 (HTTPS) Used when you activate a license online and when the system reports license status. Without it, activate offline instead -- see Storage System Activate License Offline and License Management.
Upgrades (QuantaStor packages) packages.osnexus.com TCP 80 (HTTP) The QuantaStor package repository the Upgrade Manager installs from.
Upgrades and security updates (operating system) archive.ubuntu.com, us.archive.ubuntu.com, security.ubuntu.com TCP 80 (HTTP) The Ubuntu package mirrors for the base operating system. Where appliances have no Internet access, use a local update mirror instead.
Sending support logs jira.osnexus.com TCP 21 (FTP, passive) The first upload attempt. The log bundle is encrypted before it leaves the system.
Sending support logs (fallback) services.osnexus.com TCP 443 (HTTPS) Used when FTP is unreachable. When the system has an HTTP proxy configured, QuantaStor skips FTP and uploads over HTTPS through the proxy.
Sending support logs (collection list) qstor-downloads.s3.amazonaws.com TCP 80 (HTTP) Fetches the latest list of what to collect before building the bundle.
Email alerts Your SMTP server The port you configure Set in the Alert Manager. See Call-home / Alerting.
Alert integrations The service's own endpoint, for example Slack, PagerDuty or OpsGenie TCP 443 (HTTPS) Only for the integrations you configure.
Time synchronization Your NTP servers, or ntp.ubuntu.com when none are set UDP 123 Set NTP servers per system or for the whole grid -- see Where network ports are managed.
Name resolution Your DNS servers UDP and TCP 53 Every external site on this list is reached by name.
Encryption key server Your KMIP key server TCP 5696 by default Only when a key server profile is configured.
Cloud backup and cloud containers The cloud provider's endpoint, such as s3.amazonaws.com TCP 443 (HTTPS) Only for the cloud containers and backup policies you configure.
Remote replication The other appliance's address TCP 22, TCP 5151, and TCP 20000-24999 for unencrypted links See Scale-out, clustering and replication above.

Locking down the network

Use two layers. The per-port Firewall tab and Firewall Management block services by destination address on the appliance itself; your network firewall handles everything those settings have no entry for, and all outbound traffic.

  • Keep open between grid members: TCP 5151. Add TCP 22 between systems that replicate to each other, the Corosync ranges between HA and site cluster members, and the Ceph ports between scale-out cluster members.
  • Keep open from administrator networks: TCP 443 for the web interface, TCP 22 if you use SSH, and TCP 8153 if anything calls the REST API.
  • Keep open from clients: only the storage protocols they use -- iSCSI, NVMe-oF, NFS, SMB or S3 -- and only on the ports that serve them.
  • Safe to close if unused: TCP 80 and 8080, the monitoring ports, NFS Ganesha, and every storage protocol you do not serve.

Block per port rather than system-wide where you can. A storage port that only serves iSCSI should block NFS, SMB and the web interface; a management port should block the storage protocols. After locking down, run the Network Check and send a test alert to confirm nothing you rely on was cut off.

Verifying connectivity

Navigation: Storage Management → Storage Systems → select a Storage System → Health Checker (toolbar)

The Network Check report tests reachability from each of the system's ports, including whether jumbo frames get through end to end, and reports the round-trip time for each destination. Run it after changing an MTU, since an MTU mismatch is exactly the kind of fault that leaves a port looking healthy while large transfers stall. See Network Check View.

The CLI equivalent is qs net-check:

qs net-check

Command line reference

Every operation on this page has a CLI equivalent. Add --verbose to any command to see its full argument list.

Command Purpose
qs np-list / qs np-get List all ports, or show one in detail.
qs np-modify Change addressing, MTU, portal flags, autotuning, lossless networking and the per-port firewall.
qs np-enable / qs np-disable Bring a port online or offline.
qs np-restart / qs np-rescan Restart one port, or rediscover all ports.
qs np-rename / qs np-hide Rename a port, or hide it from the interface.
qs bond-create / qs bond-delete Create and delete bonded ports.
qs vlan-create / qs vlan-delete Create and delete VLAN interfaces.
qs vif-create / qs vif-delete Create and delete virtual interfaces.
qs sr-create / qs sr-delete / qs sr-list / qs sr-get Manage static routes.
qs hai-create / qs svr-create Create HA pool and site cluster virtual interfaces.
qs net-check Run the network reachability and jumbo frame report.

Note that the --port-type argument of qs np-modify documents only static and dhcp. To remove a port's configuration entirely, use the Config Type setting of disabled in the web interface.

Set the ARP Filtering policy in the web interface, as described in ARP Filtering policy.

Related pages


Verified against QuantaStor 6.9.0.