Developing Service Containers

From OSNEXUS Online Documentation Site
Revision as of 17:49, 12 October 2026 by Qadmin (talk | contribs) (QSTOR-12896: new guide for developing universal Service Containers)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

A Service Container runs a Docker workload on a QuantaStor system and ties its lifecycle to a Resource Group: the container starts on the node that owns the Resource Group's storage pool, sees the group's network shares as directories, answers on the group's network address, and moves with the pool on HA failover. This page explains how to write a new Service Container -- the service definition file that declares its WUI controls, the handler script that starts it, how to install it, and the published examples to start from.

Section Purpose
How a Service Container runs The objects involved and what QuantaStor does with them
Files that make up a Service Container The two files you write and where they are installed
The service definition file Every [global] key, including resource limits
WUI controls The seven control types and their attributes, with how each value reaches the handler
The handler script The command line contract, the shared library, and customizing docker run
Installing a Service Container Copying the files onto every node and confirming discovery
Creating and running a service Service Config, Resource Group and shares, in the WUI and the CLI
Changing options and troubleshooting Applying option changes, logs, and common failures
Example: Web Share Browser A complete, tested plugin that uses every control type
Published examples on GitHub Downloadable handlers and how to install them

How a Service Container runs

Four objects take part:

  • A Service Definition describes a kind of service: its Docker image, the WUI controls an administrator fills in, and the handler script that starts it. QuantaStor reads one from each qs_containerhandler_<tag>.conf file. qs service-def-list lists the definitions a system has loaded.
  • A Service Config is one configured instance of a definition -- the values entered in the WUI controls.
  • A Resource Group with the Service Container Management use case binds Service Configs to a storage pool, a network address and a set of network shares.
  • A Service is the running container. QuantaStor names it <tag>-<first 6 characters of the Service Config id>-<first 6 characters of the Resource Group id>, for example webdemo-124500-3a32b3.

When a Service Config is in a Resource Group that has at least one network share, the container manager on the node that owns the pool:

  1. Writes the Service Config's non-hidden option values to the options file (see config_options_output_file).
  2. Writes each uploaded file to disk (see upload_field).
  3. Bind-mounts each of the group's network shares under <pool mount path>/.containers/<Resource Group id>/<tag>/<Service Config id>/shares/<share name>.
  4. Pulls the image if it is missing or was last pulled more than a week ago.
  5. Runs the handler script's run-container step, then its postrun-container step.

Afterwards the manager keeps the container running: if it disappears it is started again within a few minutes, and on an HA failover it is started on the node that takes over the pool. On the reference system a failover moved a running service in 42 seconds. Because everything under the Service Config's directory lives on the pool, anything the handler keeps there -- generated configuration, uploaded certificates, application data on a share -- moves with it.

Files that make up a Service Container

File Installed to Purpose
qs_containerhandler_<tag>.conf /opt/osnexus/quantastor/conf/ The service definition: image, share mount point, resource limits and WUI controls
qs_containerhandler_<tag>.sh /opt/osnexus/quantastor/bin/ The handler script that runs docker run and any post-start steps

<tag> is a short unique name such as webdemo. It must be the same in both file names and in the tag= key, or the definition is skipped with a warning in the service log. Every handler sources the shared library /opt/osnexus/quantastor/bin/qs_containerlib.sh, which QuantaStor installs; do not copy it into your plugin.

QuantaStor ships several definitions. Valkey, RabbitMQ, Nextcloud and SFTP are installed active. FTP, MinIO and Globus are installed as qs_containerhandler_<tag>.conf.example; QuantaStor only reads files ending in .conf, so rename one to enable it. These files are a good reference for real-world handlers.

The service definition file

The file has one [global] section. Write values without quotes -- they are used verbatim.

Key Required Description
type Yes Always universal.
name Yes Name shown in the Service Type list of the Service Config Create dialog.
tag Yes The service tag; must match the file names.
container_type Yes Always docker.
container_image Yes The image, passed to the handler as --containerimage. Pin an exact tag such as nginxinc/nginx-unprivileged:1.27-alpine: QuantaStor re-pulls the image weekly, so a floating tag such as latest can change the software under a running service without warning, and the two nodes of an HA pair can end up on different builds.
map_shares_dest Yes Directory inside the container where the shares appear, one subdirectory per share, for example /data/. Passed to the handler as --bindsharesdest.
container_handler_script No Path of the handler. QuantaStor runs /opt/osnexus/quantastor/bin/qs_containerhandler_<tag>.sh, so set this to that path or leave it out.
config_options_output_file When controls are declared Where the option values are written for the handler to read. {CCID} is replaced with the Service Config id, for example /var/run/quantastor/containers/webdemo_config_options.{CCID}.conf. Each non-hidden option is written as key=value, with the control's default filled in for an empty value. A definition that declares controls but omits this key fails to load.
container_memory No Memory limit, passed to docker run --memory (and --memory-swap, so the container cannot grow into swap), for example 256m or 2g.
container_cpus No CPU limit in cores, passed to docker run --cpus, for example 0.5.
container_pids_limit No Maximum processes and threads, passed to docker run --pids-limit.

Resource limits

The three limit keys stop a runaway container from starving the storage services. A key you leave out falls back to the grid-wide default in the [global] section of /opt/osnexus/quantastor/conf/qs_containerservice.conf, which ships with container_memory=8g, container_cpus=2.0 and container_pids_limit=512. Remove a key from both files to leave that resource unlimited. QuantaStor logs a warning once per service when a memory limit exceeds half the host's RAM, a CPU limit exceeds half its cores, or a PID limit exceeds 16384.

Size the limits for the workload rather than accepting the defaults. Where an application has its own memory setting (a cache size, a heap), set it to roughly 60-70% of container_memory so the application starts shedding load before the kernel kills the container.

WUI controls

Each control is declared with config_option:<key>:<attribute>=<value> lines. <key> is the option name the handler receives; use lower case letters, digits and underscores. Every control needs a type and a label.

The Service Config Create dialog for the Web Share Browser example, showing an info note, string, uint, boolean, password and upload controls, and two fieldsets.

Attributes

Attribute Applies to Description
type All One of the control types below.
label All Text beside the control. End it with a colon to match the rest of the WUI. Not displayed for info.
index All Position in the dialog, lowest first. Give every control a unique index; two controls with the same index collide.
default string, uint, boolean Value the control starts with. Also written to the options file when the value is empty.
mandatory string, uint, password, upload true to refuse an empty value. The WUI enforces it in the dialog; QuantaStor enforces it again before starting the container, so a CLI or REST caller that leaves out a mandatory option gets a container that does not start.
description All For info, the HTML that is displayed. For the other types it is not displayed in the WUI; use it to document the option for the next developer.
range_min, range_max uint Smallest and largest accepted value. With no default, the field starts at range_min.
range_min, range_max fieldset The first and last index of the controls drawn inside the fieldset.
hidden All true for credentials. See Hidden options.
custom_options string uppercase to upper-case the value written to the options file.

Control types

Type Renders as Value the handler receives
string_field Text field The text. Nothing is sent when it is empty.
password_field Masked field with a show/hide button The text. Nothing is sent when it is empty.
uint_field Integer field, validated against range_min/range_max The number.
boolean_field Check box true or false, always sent.
upload_field File chooser with a preview area The path of the uploaded file on the appliance.
fieldset A titled group around the controls whose indexes fall in its range Nothing; layout only.
info The description, rendered as HTML Nothing; use it for setup notes and links.
Every loaded Service Definition appears in the Service Type list by its name.

string_field

config_option:site_title:index=1
config_option:site_title:type=string_field
config_option:site_title:label=Site Title:
config_option:site_title:default=QuantaStor Share Browser
config_option:site_title:mandatory=true
config_option:site_title:description=Heading shown at the top of every directory listing.

Add custom_options=uppercase for values that are conventionally upper case, such as a Kerberos realm. It upper-cases the copy written to the options file only; the handler's --options list carries the value as typed, so upper-case it in the handler too if you read it from there.

password_field

config_option:auth_pass:index=6
config_option:auth_pass:type=password_field
config_option:auth_pass:hidden=true
config_option:auth_pass:label=Password:

Always pair a password with hidden=true.

uint_field

config_option:http_port:index=2
config_option:http_port:type=uint_field
config_option:http_port:label=Port:
config_option:http_port:default=9180
config_option:http_port:range_min=1024
config_option:http_port:range_max=65535
config_option:http_port:mandatory=true

boolean_field

config_option:directory_listing:index=3
config_option:directory_listing:type=boolean_field
config_option:directory_listing:label=Directory Listing:
config_option:directory_listing:default=true

upload_field

config_option:tls_pem:index=9
config_option:tls_pem:type=upload_field
config_option:tls_pem:label=TLS Certificate:

Use an upload for anything multi-line or binary-unsafe: certificates, keys, licence files. The WUI sends the file base64 encoded; QuantaStor decodes it, writes it to <pool mount path>/.containers/<Resource Group id>/<tag>/<Service Config id>/uploads/<key> with mode 0600, and passes the handler that path instead of the contents. The uploads directory is a sibling of shares, so the file is never visible through a share, it moves with the pool on failover, and its contents never appear on a command line or in the service log. To set an upload from the CLI, pass <key>=base64:$(base64 -w0 <file>).

fieldset

config_option:advanced_group:index=7
config_option:advanced_group:type=fieldset
config_option:advanced_group:label=Advanced Settings
config_option:advanced_group:range_min=8
config_option:advanced_group:range_max=9

The fieldset is drawn at its own index and contains the controls with indexes 8 and 9. Number the contained controls immediately after the fieldset so the order in the file matches the order on screen.

info

config_option:about:index=0
config_option:about:type=info
config_option:about:label=About:
config_option:about:description=<div style="padding:6px 8px;border:1px solid #9ab8d8;background:#eef4fb;border-radius:4px;">Publishes the network shares ... See the <a href="https://wiki.osnexus.com/index.php/Developing_Service_Containers" target="_blank">guide</a>.</div>

Keep the HTML on one line. Use an info note at index 0 for prerequisites the administrator must complete before creating the service, with a link to your setup guide; the Globus definition does this.

Hidden options

An option with hidden=true is handled as a credential:

  • It is passed to the handler in a separate --options-hidden argument, and the copy of the command line written to the service log shows <redacted> in its place. qs_containerlib.sh folds it back into $OPTIONS, so the handler reads it like any other option.
  • It is not written to the options file.

It is not encrypted: the value is stored in the configuration database, and qs service-config-options-list displays it. Treat it accordingly, and log your own docker run command line through redact_cmd (see below) so the handler does not leak it either.

How values reach the handler

The handler receives every set option in one comma-separated list, --options "key:value,key:value", plus --options-hidden for hidden ones. Three consequences:

  • A value cannot contain a comma, a newline or a tab -- the list is split on them, and an = in a CLI value is read as the key separator. Use an upload_field for anything that might.
  • Empty string, password and upload options are left out of the list entirely, and defaults are not added to it. Give every option a default in the handler as well as in the definition.
  • An option key the definition does not declare is refused when a Service Config is created or changed.

If you prefer reading a file, the same values are in the options file named by config_options_output_file, as key=value lines with defaults filled in and hidden options left out. It is under /var/run, which is cleared at boot, and it is rewritten before every start.

The handler script

The container manager runs the handler twice per start, as root, on the node that owns the pool: 

qs_containerhandler_<tag>.sh run-container \
    --container <container name> --bindaddrs <ip[,ip...]> \
    --bindsharessrc <host shares directory> --bindsharesdest <map_shares_dest> \
    --containerimage <container_image> --confpath <options file> --ccid <Service Config id> \
    --options "key:value,..." [--options-hidden "key:value,..."] \
    [--memlimit <m>] [--cpulimit <c>] [--pidslimit <n>]
qs_containerhandler_<tag>.sh postrun-container <same arguments>
  • run-container must start the container and print only the output of docker run -d (the container id) on stdout. A non-zero exit status marks the start as failed. Write all messages to the service log with logservice, which writes to stderr.
  • postrun-container runs after a successful start. Use it for readiness checks or configuration that needs the container running. Its exit status is ignored, so log any problem it finds.

The shared library

Source qs_containerlib.sh from the handler's own directory, then end the script with its two entry points: 

LIB_DIR="$(cd -- "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
source "${LIB_DIR}/qs_containerlib.sh"
...
parse_cli OPERATION "$@"
dispatch "$OPERATION" run-container postrun-container usage version

parse_cli sets these globals:

Variable Contents
CONTAINERNAME Container name to pass to docker run --name
BINDADDR The Resource Group's network address(es), comma-separated
BINDSRC Host directory holding one subdirectory per bound share
BINDDST map_shares_dest
CONTAINERIMAGE container_image
CONFPATH The options file
CCID The Service Config id
OPTIONS All options, hidden ones included, as key:value,...
OPTIONS_HIDDEN The hidden options on their own
RESLIMIT_ARGS Ready-made --memory/--cpus/--pids-limit arguments; empty when no limit is set
SERVICEUSER_ARGS -u <uid:gid> when the handler sets SERVICEUSER before sourcing the library; empty otherwise

and provides these helpers:

Function Use
logservice LEVEL message Append a message to /var/log/qs/qs_service.log
redact_cmd args... Return a command line with credential-looking values (*PASS*=, *SECRET*=, *TOKEN*=, *KEY*=, --password <x> and similar) replaced by <redacted>, for logging
container_share_datadir In-container path of the share that holds the service's data: the share named by a data_share option if one is set and bound, otherwise the first bound share alphabetically
container_share_datadir_host The same directory as a host path, for the handler to prepare before the container starts
prepare_service_datadir <dir> Create a directory and give it to SERVICEUSER -- that directory only, not recursively

Customizing startup

run-container is where your plugin's behaviour lives. A typical handler:

  1. Parses $OPTIONS into variables, applying a default for each.
  2. Validates them and exits non-zero with a logservice "ERROR" message if the configuration cannot work.
  3. Renders the application's own configuration files from the options into a private directory next to shares -- "$(dirname "${BINDSRC%/}")/runtime" -- so they follow the pool and are never visible through a share.
  4. Builds the docker run command as a bash array and runs it.

Build the command with these pieces:

  • --restart=always -d, so Docker restarts a crashed container.
  • "${RESLIMIT_ARGS[@]}" and "${SERVICEUSER_ARGS[@]}".
  • -p "$addr:<host port>:<container port>" for each address in $BINDADDR. Publish on the Resource Group's addresses only, never on all interfaces, so the service follows its virtual interface on failover.
  • --mount "type=bind,source=$BINDSRC,target=$BINDDST,bind-propagation=rslave". rslave lets a share added to the Resource Group later appear inside the running container. Add readonly for a service that must not modify share data.
  • -e NAME=value for environment variables, and further --mount options for the files you rendered.
  • --name "$CONTAINERNAME" and "$CONTAINERIMAGE", followed by any command or arguments the image needs.

Log the command with logservice "INFO" "Running: $(redact_cmd "${DOCKER_CMD[@]}")" before running it.

Rules for a safe handler

  • Never publish a port the appliance already uses. The web manager listens on 80, 443 and 8080, and storage and management services use others (22, 111, 139, 445, 2049, 3260, 5151-5156, 8086, 8088, 8153, 8154 among them). Docker reports such a conflict as driver failed programming external connectivity, and QuantaStor responds to that error by restarting the Docker daemon, which stops every other container on the node -- repeated quickly enough, systemd stops restarting Docker at all. Check the port in the handler before running the container, as the example below does with ss, and choose defaults above 9000.
  • Check the image's entrypoint before shipping. Many images recursively chown their data directory when started as root (valkey and rabbitmq both do). Every bound share is mounted inside that directory, so such an image rewrites the ownership of customer files. Set SERVICEUSER so the image takes its non-root path, and call prepare_service_datadir for the one directory it needs. Inspect an image with: 
    docker run --rm --entrypoint sh <image> -c 'cat /docker-entrypoint.sh' | grep -nE 'chown|find \.|rsync|--delete|rm -rf'
  • Never make map_shares_dest an application's install or document root. An image that initializes its directory with rsync --delete (Nextcloud does) would delete everything in the bound shares on first start. Mount the shares elsewhere and give the application a private subdirectory.
  • Prefer images that run unprivileged, such as the nginx-unprivileged image the example uses.

Installing a Service Container

Install the two files on every node that can own the pool -- both nodes of an HA pair -- or the service cannot start after a failover. 

cp qs_containerhandler_webdemo.conf /opt/osnexus/quantastor/conf/
cp qs_containerhandler_webdemo.sh /opt/osnexus/quantastor/bin/
chown root:root /opt/osnexus/quantastor/conf/qs_containerhandler_webdemo.conf /opt/osnexus/quantastor/bin/qs_containerhandler_webdemo.sh
chmod 0644 /opt/osnexus/quantastor/conf/qs_containerhandler_webdemo.conf
chmod 0755 /opt/osnexus/quantastor/bin/qs_containerhandler_webdemo.sh
systemctl restart quantastor

QuantaStor reads service definitions when its service starts, and pulls each definition's image at the same time. After the restart, confirm the definition loaded: 

qs service-def-list
grep webdemo /var/log/qs/qs_service.log

A definition that does not appear is explained in the service log -- typically a tag that does not match the file name, a missing required key, or a missing handler script.

Changes to the handler script take effect at the next container start; changes to the definition file need another restart of the quantastor service.

Creating and running a service

Create the Service Config

Navigation: Multitenancy → Resource Groups (section) → Services (toolbar group) → Create Config (toolbar)

Enter a name, choose your definition in Service Type, and fill in the controls. The Create Resilio Config and Create FTP Config buttons in the Services section open the same dialog with a different type selected.

CLI equivalent: qs service-config-create --name=webshare --service-definition-tag=webdemo --service-config-options-list=site_title=Lab,http_port=9180

Create the Resource Group

Choose the Service Container Management use case when creating the Resource Group.
Navigation: Multitenancy → Resource Groups (section) → Resource Group (toolbar group) → Create (toolbar)
  1. On the General tab, enter a name and select Service Container Management.
  2. On the Service Configs tab, choose the Subnet w/CIDR the service answers on, the Storage Pool Scope -- HA Storage Pool for a pool in an HA group, so the service fails over with it -- and the Storage Pool, then tick the Service Config.
  3. Click OK.

The subnet must contain an address on the pool's node; for an HA pool, the pool's virtual interface.

Add network shares

Add the network shares the service should see.
Navigation: Multitenancy → Resource Groups (section) → select the Resource Group → Resource Group (toolbar group) → Add/Remove Shares (toolbar)

Move one or more shares from the pool to the right-hand list, click OK and confirm. The container starts once the group has a share; a Resource Group with no share does not start its services.

CLI equivalent: qs resource-group-resource-add --resource-group=webshare-rg --resource-list=share-1:share

Check the service

The running container in the Services section.
Navigation: Multitenancy → Services (section)

The Services section lists each running container with its Service Config and definition. qs service-list shows the same, and docker ps on the node that owns the pool shows the container itself.

Changing options and troubleshooting

Changing options

  • qs service-config-option-set changes options the Service Config already has. An option that was never set -- an empty field, an upload left blank -- must be added with qs service-config-option-add instead.
  • Changed options are stored at once but take effect at the next start of the container; a running container is not restarted. While developing, remove the container on the node that owns the pool with docker rm -f <container name>; QuantaStor starts it again, with the new options, within a few minutes.

Where to look

  • /var/log/qs/qs_service.log on the node that owns the pool: the exact handler command line (hidden options redacted), every logservice message from your handler, and Docker's error if docker run failed.
  • journalctl -u docker for errors from the Docker daemon.
  • To test a handler without QuantaStor, run it with the same arguments the service log shows, pointing --bindsharessrc at any directory, and add --verbose.

Common failures

Symptom Cause
Definition missing from the Service Type list The file does not end in .conf, the tag does not match the file name, a required key is missing, the handler is not at /opt/osnexus/quantastor/bin/qs_containerhandler_<tag>.sh, or the quantastor service has not been restarted since the file was added.
Docker restarted, other containers stopped, or Docker left stopped A published port is already in use; see Rules for a safe handler. Run systemctl reset-failed docker; systemctl start docker to recover.
Container never starts The Resource Group has no network share, a mandatory option is missing, or the handler exited non-zero; the service log says which.
An option has no effect The container was not restarted after the change, or the option was empty and so was never sent to the handler.
Files on a share changed owner The image chowns its data directory as root; see Rules for a safe handler.

Example: Web Share Browser

A share listed by the Web Share Browser, with the site title and upper-cased environment tag.

This example publishes a Resource Group's shares as a read-only web directory listing using the unprivileged nginx image. It is small enough to read in one sitting, and it uses every control type plus the common handler techniques: option defaults, validation, a rendered configuration file, an HTTP password file, an uploaded TLS certificate, a read-only share mount, resource limits and a post-start readiness check. It has been run on an HA pair, including a failover.

To try it, save the two files below, install them, create a Service Config of type Web Share Browser (Example), and add it with a share to a Resource Group. Browse to http://<Resource Group address>:9180/, or https:// once a certificate is uploaded. The certificate upload takes a single PEM file holding the certificate followed by its private key.

qs_containerhandler_webdemo.conf

[global]
# Example 'universal' service container: a read-only web browser for the network shares bound to a
# Resource Group. It is deliberately small and uses every WUI control type a service definition can
# declare, so it doubles as the reference for writing new service containers. It ships as
# qs_containerhandler_webdemo.conf.example; rename it to .conf to enable it.
type=universal
# Friendly name shown in the Service Type list of the Create Service Config dialog
name=Web Share Browser (Example)
# Container framework
container_type=docker
# Container image, pinned to an exact tag so every node of an HA pair runs the same build
container_image=nginxinc/nginx-unprivileged:1.27-alpine
# Service tag, must match the <tag> in the file name qs_containerhandler_<tag>.conf
tag=webdemo
# Where the Resource Group's shares appear inside the container, one subdirectory per share.
# NOTE: values here are used verbatim, do not quote them.
map_shares_dest=/data/
# Script that runs the container ('docker run') and any post-run steps
container_handler_script=/opt/osnexus/quantastor/bin/qs_containerhandler_webdemo.sh
# Where the non-hidden option values are written for the handler to read, {CCID} is replaced with
# the Service Config id. Required whenever config_option entries are declared.
config_options_output_file=/var/run/quantastor/containers/webdemo_config_options.{CCID}.conf

# cgroup limits for each instance; omit a key to inherit the [global] default in
# qs_containerservice.conf
container_memory=256m
container_cpus=0.5
container_pids_limit=128

# Options rendered in the WUI, in 'index' order. Each needs a 'type' and a 'label'.
config_option:about:index=0
config_option:about:type=info
config_option:about:label=About:
config_option:about:description=<div style="padding:6px 8px;margin-bottom:8px;border:1px solid #9ab8d8;background:#eef4fb;border-radius:4px;">Publishes the network shares in this Service Config's Resource Group as a read-only web directory listing on the Resource Group's network address. See the <a href="https://wiki.osnexus.com/index.php/Developing_Service_Containers" target="_blank">Developing Service Containers</a> guide.</div>

config_option:site_title:index=1
config_option:site_title:type=string_field
config_option:site_title:label=Site Title:
config_option:site_title:default=QuantaStor Share Browser
config_option:site_title:mandatory=true
config_option:site_title:description=Heading shown at the top of every directory listing.

config_option:http_port:index=2
config_option:http_port:type=uint_field
config_option:http_port:label=Port:
config_option:http_port:default=9180
config_option:http_port:range_min=1024
config_option:http_port:range_max=65535
config_option:http_port:mandatory=true
config_option:http_port:description=Port published on the Resource Group's network address(es). Serves HTTPS when a TLS certificate is uploaded, HTTP otherwise. Must not be a port the appliance already uses, such as 80, 443, 8080 or 8443.

config_option:directory_listing:index=3
config_option:directory_listing:type=boolean_field
config_option:directory_listing:label=Directory Listing:
config_option:directory_listing:default=true
config_option:directory_listing:description=List directory contents. When off, only files whose exact path is known can be downloaded.

config_option:access_group:index=4
config_option:access_group:type=fieldset
config_option:access_group:label=Access Control
config_option:access_group:range_min=5
config_option:access_group:range_max=6

config_option:auth_user:index=5
config_option:auth_user:type=string_field
config_option:auth_user:label=Username:
config_option:auth_user:description=Optional. When set together with a password, every request needs HTTP basic authentication.

config_option:auth_pass:index=6
config_option:auth_pass:type=password_field
config_option:auth_pass:hidden=true
config_option:auth_pass:label=Password:
config_option:auth_pass:description=Password for the user above. Hidden, so it is kept out of the service log and the options file.

config_option:advanced_group:index=7
config_option:advanced_group:type=fieldset
config_option:advanced_group:label=Advanced Settings
config_option:advanced_group:range_min=8
config_option:advanced_group:range_max=9

config_option:env_tag:index=8
config_option:env_tag:type=string_field
config_option:env_tag:label=Environment Tag:
config_option:env_tag:custom_options=uppercase
config_option:env_tag:description=Optional short tag such as PROD or DR, shown beside the title. Stored in upper case.

config_option:tls_pem:index=9
config_option:tls_pem:type=upload_field
config_option:tls_pem:label=TLS Certificate:
config_option:tls_pem:description=Optional PEM file holding the server certificate followed by its private key. Switches the port to HTTPS.

qs_containerhandler_webdemo.sh

#!/usr/bin/env bash
# qs_containerhandler_webdemo.sh - Web Share Browser example service container handler
#
# Reference handler for writing new universal service containers. It shows the usual steps:
# parse the WUI options, validate them, render the application's own configuration from them,
# build the 'docker run' line (port publishing, share bind mount, resource limits) and check the
# service from postrun-container.

##############################################################################
# ------------------- script-specific constants ---------------------------- #
##############################################################################
CONTAINERTAG="webdemo"
CONTAINERDESC="Web Share Browser Example Container Handler"
VERSION="v1.0"
COPYRIGHT="Copyright (c) 2026 OSNexus Corporation"

APPNAME="qs_containerhandler_${CONTAINERTAG}"
APPDESC="$APPNAME $VERSION - $CONTAINERDESC"

# The image runs nginx as uid/gid 101 and listens on this port inside the container
NGINX_UID="101"
CONTAINER_PORT="8080"

##############################################################################
# ---------------- import container handler shared functions --------------- #
##############################################################################
LIB_DIR="$(cd -- "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
# shellcheck source=qs_containerlib.sh
source "${LIB_DIR}/qs_containerlib.sh"

##############################################################################
# -------------------------- container specific logic ---------------------- #
##############################################################################

usage()
{
  printf "\nExamples:\n\n"
  printf "    %s.sh run-container -c webdemo-test -b 10.0.0.10 -s /mnt/storage-pools/qs-<id>/.containers/<rg>/webdemo/<ccid>/shares -d /data/ -i nginxinc/nginx-unprivileged:1.27-alpine -x <ccid> -o \"site_title:Lab Shares,http_port:8080,directory_listing:true\"\n" "$APPNAME"
  printf "\n"
}

# Parse the comma-delimited key:value OPTIONS list (hidden options are already folded in by
# qs_containerlib.sh) into globals, applying the same defaults as the service definition.
parse_options()
{
  SITE_TITLE="QuantaStor Share Browser"
  HTTP_PORT="9180"
  DIRECTORY_LISTING="true"
  AUTH_USER=""
  AUTH_PASS=""
  ENV_TAG=""
  TLS_PEM=""

  local opt
  local opts
  IFS=',' read -ra opts <<< "$OPTIONS"
  for opt in "${opts[@]}"; do
    case "$opt" in
      site_title:*)        SITE_TITLE="${opt#site_title:}" ;;
      http_port:*)         HTTP_PORT="${opt#http_port:}" ;;
      directory_listing:*) DIRECTORY_LISTING="${opt#directory_listing:}" ;;
      auth_user:*)         AUTH_USER="${opt#auth_user:}" ;;
      auth_pass:*)         AUTH_PASS="${opt#auth_pass:}" ;;
      env_tag:*)           ENV_TAG="${opt#env_tag:}" ;;
      tls_pem:*)           TLS_PEM="${opt#tls_pem:}" ;;   # upload_field: path of the uploaded file
    esac
  done

  # The title ends up inside the nginx configuration, keep it to characters that need no quoting
  SITE_TITLE="$(printf '%s' "$SITE_TITLE" | tr -cd 'A-Za-z0-9 ._()-')"
  # custom_options=uppercase is applied to the options file only, OPTIONS carries the value as typed
  ENV_TAG="$(printf '%s' "${ENV_TAG^^}" | tr -cd 'A-Z0-9_-')"
}

# Refuse configurations that cannot work, so the failure is reported on the Service Config
# instead of leaving a container that restarts forever.
validate_options()
{
  if ! [[ "$HTTP_PORT" =~ ^[0-9]+$ ]] || (( HTTP_PORT < 1024 || HTTP_PORT > 65535 )); then
    logservice "ERROR" "Port '$HTTP_PORT' is not in the range 1024-65535."
    return 1
  fi
  # A port the appliance already listens on (the web manager owns 8080, for one) makes 'docker run'
  # fail with "driver failed programming external connectivity", which the container manager treats
  # as a Docker fault and answers by restarting dockerd. Refuse it here so that never happens.
  local addr
  local addrs
  IFS=', ' read -r -a addrs <<< "$BINDADDR"
  for addr in "${addrs[@]}" "0.0.0.0" "*" "[::]"; do
    [[ -z "$addr" ]] && continue
    if ss -ltnH "sport = :$HTTP_PORT" 2>/dev/null | awk '{print $4}' | grep -qxF "$addr:$HTTP_PORT"; then
      logservice "ERROR" "Port $HTTP_PORT is already in use on $addr, choose another port."
      return 1
    fi
  done
  if [[ -n "$AUTH_USER" && -z "$AUTH_PASS" ]] || [[ -z "$AUTH_USER" && -n "$AUTH_PASS" ]]; then
    logservice "ERROR" "Basic authentication needs both a username and a password."
    return 1
  fi
  if [[ -n "$TLS_PEM" ]]; then
    if ! openssl x509 -noout -in "$TLS_PEM" >/dev/null 2>&1 || ! openssl pkey -noout -in "$TLS_PEM" >/dev/null 2>&1; then
      logservice "ERROR" "The uploaded TLS certificate must be a PEM file holding a certificate and its private key."
      return 1
    fi
  fi
  return 0
}

# Private per-instance directory for generated configuration. It sits beside the 'shares' staging
# directory on the pool, so it follows the pool on HA failover and is never visible through the
# share bind mount.
runtime_dir()
{
  printf '%s/runtime' "$(dirname "${BINDSRC%/}")"
}

# Render the nginx server block, and the htpasswd and certificate files it references
render_config()
{
  local rundir="$1"
  mkdir -p "$rundir" && chmod 0755 "$rundir" || return 1

  local heading="$SITE_TITLE"
  [[ -n "$ENV_TAG" ]] && heading="$heading [$ENV_TAG]"

  local listen="$CONTAINER_PORT"
  local tls_lines=""
  if [[ -n "$TLS_PEM" ]]; then
    install -m 0600 -o "$NGINX_UID" -g "$NGINX_UID" "$TLS_PEM" "$rundir/server.pem" || return 1
    listen="$CONTAINER_PORT ssl"
    tls_lines="    ssl_certificate     /etc/nginx/qs/server.pem;
    ssl_certificate_key /etc/nginx/qs/server.pem;"
  else
    rm -f "$rundir/server.pem"
  fi

  local auth_lines=""
  if [[ -n "$AUTH_USER" ]]; then
    printf '%s:%s\n' "$AUTH_USER" "$(printf '%s' "$AUTH_PASS" | openssl passwd -apr1 -stdin)" > "$rundir/htpasswd" || return 1
    chown "$NGINX_UID:$NGINX_UID" "$rundir/htpasswd" && chmod 0600 "$rundir/htpasswd"
    auth_lines="    auth_basic           \"$SITE_TITLE\";
    auth_basic_user_file /etc/nginx/qs/htpasswd;"
  else
    rm -f "$rundir/htpasswd"
  fi

  local autoindex="off"
  [[ "$DIRECTORY_LISTING" == "true" ]] && autoindex="on"

  cat > "$rundir/default.conf" <<EOF
# Generated by $APPNAME on every start, edits are overwritten
server {
    listen $listen;
$tls_lines
$auth_lines
    root /data;
    autoindex $autoindex;
    autoindex_exact_size off;
    autoindex_localtime on;
    sub_filter '<h1>' '<h1>$heading &mdash; ';
    sub_filter '<title>' '<title>$heading &mdash; ';
    sub_filter_once off;
    location ~ /\.  { deny all; }
}
EOF
  chmod 0644 "$rundir/default.conf"
}

# Required: start the container. The container manager treats stdout as the new container id,
# so only 'docker run' may write to it; log through logservice.
run-container()
{
  parse_options
  validate_options || return 1

  local rundir
  rundir="$(runtime_dir)"
  render_config "$rundir" || { logservice "ERROR" "Could not write the configuration to '$rundir'."; return 1; }

  local addrs
  IFS=', ' read -r -a addrs <<< "$BINDADDR"

  DOCKER_CMD=(/usr/bin/docker run --restart=always -d)
  DOCKER_CMD+=("${RESLIMIT_ARGS[@]}")

  # Publish the port only on the Resource Group's addresses, never on every host interface
  local addr
  for addr in "${addrs[@]}"; do
    [[ -z "$addr" ]] && continue
    DOCKER_CMD+=(-p "$addr:$HTTP_PORT:$CONTAINER_PORT")
  done

  # The shares, read-only; rslave so a share added to the Resource Group later appears without a restart
  if [[ -n "$BINDSRC" && -n "$BINDDST" ]]; then
    DOCKER_CMD+=(--mount "type=bind,source=$BINDSRC,target=${BINDDST%/},readonly,bind-propagation=rslave")
  fi
  # The generated configuration
  DOCKER_CMD+=(--mount "type=bind,source=$rundir/default.conf,target=/etc/nginx/conf.d/default.conf,readonly")
  DOCKER_CMD+=(--mount "type=bind,source=$rundir,target=/etc/nginx/qs,readonly")

  [[ -n "$CONTAINERNAME" ]] && DOCKER_CMD+=(--name "$CONTAINERNAME")
  DOCKER_CMD+=("$CONTAINERIMAGE")

  logservice "INFO" "Running container '${CONTAINERNAME}' with command '$(redact_cmd "${DOCKER_CMD[@]}")'."
  "${DOCKER_CMD[@]}"
}

# Optional: runs after the container has started. Here it waits for nginx to accept its
# configuration and logs the address the service answers on.
postrun-container()
{
  parse_options

  for _ in $(seq 1 10); do
    if /usr/bin/docker exec "$CONTAINERNAME" nginx -t >/dev/null 2>&1; then
      local scheme="http"
      [[ -n "$TLS_PEM" ]] && scheme="https"
      logservice "INFO" "Web Share Browser '${CONTAINERNAME}' is serving ${scheme}://${BINDADDR%%,*}:${HTTP_PORT}/"
      return 0
    fi
    sleep 1
  done
  logservice "WARN" "Web Share Browser '${CONTAINERNAME}' did not pass 'nginx -t' within 10 seconds."
  return 1
}

##############################################################################
# ----------------------------- main entry --------------------------------- #
##############################################################################

parse_cli OPERATION "$@"
dispatch "$OPERATION" run-container postrun-container usage version

Published examples on GitHub

OSNEXUS solutions engineering publishes handlers for blog walkthroughs in the quantastor-blog-files repository. Each tarball holds a .conf and a .sh:

Download Service Notes
qs_containerhandler_globus.tgz Globus Connect Server Needs a Globus subscription and one-time setup; see the QuantaStor Service Containers: Globus Connect Server walkthrough and the globus-connect-server-deploy repository.
qs_containerhandler_minio.tgz MinIO S3 server Serves a bound share over S3.
qs_containerhandler_opencloud.tgz OpenCloud file sync and share

To install one, download and unpack it on each node, then follow Installing a Service Container: 

wget https://github.com/steve-jordahllabs/quantastor-blog-files/raw/refs/heads/main/qs_containerhandler_minio.tgz
tar xzf qs_containerhandler_minio.tgz

The Globus and MinIO definitions also ship with QuantaStor as .conf.example files, and the shipped versions have changed since the tarballs were published -- the shipped MinIO handler, for example, works with a Resource Group that has several shares, where the tarball's requires exactly one. Prefer the shipped version where one exists, and compare the two before using a tarball. Read any handler before installing it: it runs as root on your storage system.

The RabbitMQ bucket notifications post describes the RabbitMQ service that QuantaStor now ships as an active definition; its companion files are in ceph-notification-rabbitmq.

Related pages

Verified against QuantaStor 6.9.0.

Retrieved from "https://wiki.osnexus.com/index.php?title=Developing_Service_Containers&oldid=28254"