Qadmin: osn-seo-utilities: datadog-integration @ 7b79d754f6a9 (approved in the portal)

2026-11-03T20:00:04Z

osn-seo-utilities: datadog-integration @ 7b79d754f6a9 (approved in the portal)

New page

QuantaStor can send its alerts to [https://www.datadoghq.com/ Datadog] as Datadog events, through the Datadog module of the [[Storage System Alert Manager|Alert Manager]]. This page covers creating the Datadog API key, adding the endpoint in QuantaStor, testing it, and building Datadog monitors on the severity and system tags that every event carries.

{| class="wikitable"
! Section !! Purpose
|-
| [[#How it works|How it works]] || What is sent, where, and what is not sent
|-
| [[#Step 1: Create a Datadog API key|Step 1: Create a Datadog API key]] || The key the endpoint URL carries
|-
| [[#Step 2: Add the Datadog endpoint|Step 2: Add the Datadog endpoint]] || The Alert Manager ITSM Integrations tab
|-
| [[#Step 3: Send a test alert|Step 3: Send a test alert]] || Confirm events arrive in Datadog
|-
| [[#Step 4: Create Datadog monitors|Step 4: Create Datadog monitors]] || Turn QuantaStor events into notifications
|-
| [[#Endpoint URL format and Datadog sites|Endpoint URL format and Datadog sites]] || The URL forms accepted and the site each host selects
|-
| [[#Event contents|Event contents]] || Title, body, host and date of each event
|-
| [[#Severity mapping and tags|Severity mapping and tags]] || How QuantaStor severities and tags appear in Datadog
|-
| [[#Configuring from the CLI|Configuring from the CLI]] || Adding, removing and testing the endpoint with qs
|-
| [[#Troubleshooting|Troubleshooting]] || No events, 403 errors, missing Info alerts
|}

== How it works ==

Each QuantaStor alert of Warning, Error or Critical severity is posted directly from the appliance to the Datadog Events API as a Datadog event. No Datadog Agent is needed on the appliance. Every event carries tags that name the product, the severity, the storage system and the grid, so you can filter the Datadog Event Explorer and build monitors on them.

Three behaviours to know before you start:

* '''Info alerts are never sent.''' QuantaStor sends only Warning, Error and Critical alerts to ITSM modules, Datadog included. A test at Info level produces nothing in Datadog, which makes a correctly configured endpoint look broken.
* '''Clearing an alert sends nothing.''' Datadog events cannot be closed, so dismissing or clearing an alert in QuantaStor has no counterpart in Datadog. Use monitor recovery logic in Datadog if you need one.
* '''Repeats roll up.''' Repeats of the same alert type on the same storage system share an aggregation key, so Datadog groups them together rather than listing each one separately.

The appliance needs outbound HTTPS access to the Datadog API host for your site. If the appliance is configured to use an HTTPS proxy, the module sends through it.

== Step 1: Create a Datadog API key ==

In Datadog, open '''Organization Settings''' → '''API Keys''' and create a key for QuantaStor, or copy an existing one. Use an '''API key''', not an Application key: the Events API accepts only API keys.

Note which Datadog site your organization is on (US1, US3, US5, EU1, AP1, AP2 or US1-FED). The key works only against that site's API host; see [[#Endpoint URL format and Datadog sites|Endpoint URL format and Datadog sites]].

== Step 2: Add the Datadog endpoint ==

{{Navigation|Storage Management → Storage System ''(toolbar group)'' → Alert Manager ''(toolbar)'' → ITSM Integrations ''(tab)''}}

[[File:alertmgr_itsm_endpoints.png|thumb|right|800px|Alert Manager, ITSM Integrations tab with a Datadog endpoint added. Choose the module, paste the URL into Webhook URL, then click Add.]]

The ITSM Integrations tab lists every configured endpoint, one row per module and URL. To add Datadog:

# Open the '''Alert Manager''' and select the '''ITSM Integrations''' tab.
# In '''Module''', select '''Datadog'''. The '''?''' button beside it opens the setup page for the selected module.
# In '''Webhook URL''', paste the Datadog API host followed by {{Code|1=#}} and your API key, for example {{Code|1=https://api.datadoghq.com#aAbBcCdDeEfF0123456789aAbBcCdDeE}}.
# Click '''Add'''. The endpoint appears in the list.
# Click '''OK''' or '''Apply''' to save the configuration.

[[File:alertmgr_itsm_module_list.png|thumb|right|800px|The Module drop-down lists every ITSM module the appliance supports, Datadog among them.]]

To remove an endpoint, tick it in the list, click '''Remove Selected''', then click '''OK''' or '''Apply'''.

QuantaStor sends the API key to Datadog in the {{Code|1=DD-API-KEY}} request header, never in the request URL, and masks it as {{Code|1=********}} in the service log.

== Step 3: Send a test alert ==

[[File:Alertmgr_test_alert.png|thumb|right|328px|Send User Generated Alert, opened from Generate Test Alert. Severity offers Warning, Error and Critical.]]

Click '''Generate Test Alert''' at the bottom of the Alert Manager. The '''Send User Generated Alert''' dialog opens with two fields:

* '''Message''' -- the text of the test alert.
* '''Severity''' -- Warning, Error or Critical. Info is not offered, because Info alerts never reach ITSM modules.

Click '''OK''', then open the Datadog '''Event Explorer''' and search for {{Code|1=product:quantastor}}. The test event appears within a few seconds, with the host set to the storage system name.

If nothing arrives, see [[#Troubleshooting|Troubleshooting]].

== Step 4: Create Datadog monitors ==

Events on their own notify nobody. To page an on-call engineer, create a Datadog '''Event Monitor''' for each severity you want to act on. For example:

{| class="wikitable"
! Monitor !! Query !! Condition
|-
| QuantaStor critical alerts || {{Code|1=product:quantastor severity:critical}} || count > 0
|-
| QuantaStor error alerts || {{Code|1=product:quantastor severity:error}} || count > 0
|-
| QuantaStor warnings || {{Code|1=product:quantastor severity:warning}} || count > 0
|}

Filter on the {{Code|1=severity}} tag rather than on the Datadog event status: Datadog has no Critical status, so Error and Critical alerts both arrive with status Error, and only the tag keeps them apart. Add {{Code|1=system:<name>}} or {{Code|1=grid:<name>}} to a query to scope a monitor to one storage system or one grid, and route each monitor to the team or channel that owns it.

== Endpoint URL format and Datadog sites ==

The Webhook URL takes either of two forms:

{| class="wikitable"
! Form !! Example
|-
| API host, {{Code|1=#}}, API key || {{Code|1=https://api.datadoghq.com#aAbBcCdDeEfF0123456789aAbBcCdDeE}}
|-
| Datadog's own Events API URL || {{Code|1=https://api.datadoghq.com/api/v1/events?api_key=aAbBcCdDeEfF0123456789aAbBcCdDeE}}
|}

Both send the key in a request header; the second form exists so you can paste the URL Datadog's documentation gives.

The host selects the Datadog site:

{| class="wikitable"
! Datadog site !! API host
|-
| US1 || {{Code|1=api.datadoghq.com}}
|-
| US3 || {{Code|1=api.us3.datadoghq.com}}
|-
| US5 || {{Code|1=api.us5.datadoghq.com}}
|-
| EU1 || {{Code|1=api.datadoghq.eu}}
|-
| AP1 || {{Code|1=api.ap1.datadoghq.com}}
|-
| AP2 || {{Code|1=api.ap2.datadoghq.com}}
|-
| US1-FED || {{Code|1=api.ddog-gov.com}}
|}

If you paste a site or web interface host instead, such as {{Code|1=datadoghq.eu}} or {{Code|1=app.datadoghq.com}}, QuantaStor maps it to the matching API host automatically.

== Event contents ==

Each QuantaStor alert becomes one Datadog event:

{| class="wikitable"
! Event field !! Content
|-
| Title || The alert title, prefixed with the alert type number in brackets, for example {{Code|1=[122] ...}}
|-
| Text || The alert description, followed by Severity, Alert Time, Alert Type, Storage System (linked to its IP address), Storage System ID, Storage Grid, Location, Serial Number, Service Version, Firmware Version, Kernel Version, System Startup and Alert ID
|-
| Host || The storage system name
|-
| Date || The time QuantaStor raised the alert
|}

== Severity mapping and tags ==

{| class="wikitable"
! QuantaStor severity !! Datadog event status !! Severity tag
|-
| Info || ''not sent'' || --
|-
| Warning || Warn || {{Code|1=severity:warning}}
|-
| Error || Error || {{Code|1=severity:error}}
|-
| Critical || Error || {{Code|1=severity:critical}}
|}

Every event carries these tags:

{| class="wikitable"
! Tag !! Value
|-
| {{Code|1=product:quantastor}} || Always present; use it to select all QuantaStor events
|-
| {{Code|1=severity:<level>}} || {{Code|1=warning}}, {{Code|1=error}} or {{Code|1=critical}}
|-
| {{Code|1=system:<name>}} || The storage system that raised the alert
|-
| {{Code|1=system_id:<id>}} || The storage system's unique ID
|-
| {{Code|1=grid:<name>}} || The storage grid the system belongs to
|-
| {{Code|1=alert_type_id:<id>}} || The alert type number, the same one shown in the title
|-
| {{Code|1=location:<location>}} || The storage system's location, only when one is set
|}

Datadog adds {{Code|1=source:my_apps}} itself.

== Configuring from the CLI ==

Use <code>[[QuantaStor CLI Command Reference#alert-config-set|qs alert-config-set]]</code> with {{Code|1=--endpoints}} to add the endpoint. Each entry is {{Code|1=module:url}}; separate several entries with commas.

<pre style="font-size: smaller">
qs alert-config-set --endpoints="datadog:https://api.datadoghq.com#aAbBcCdDeEfF0123456789aAbBcCdDeE"
</pre>

To remove the endpoint, give the same entry with a {{Code|1=~}} prefix:

<pre style="font-size: smaller">
qs alert-config-set --endpoints="~datadog:https://api.datadoghq.com#aAbBcCdDeEfF0123456789aAbBcCdDeE"
</pre>

Send a test alert with <code>[[QuantaStor CLI Command Reference#alert-raise|qs alert-raise]]</code>. Its {{Code|1=--alert-severity}} defaults to {{Code|1=info}}, which never reaches Datadog, so always set it to {{Code|1=warning}}, {{Code|1=error}} or {{Code|1=critical}}:

<pre style="font-size: smaller">
qs alert-raise --title="Datadog test" --message="Testing the Datadog endpoint" --alert-severity=warning
</pre>

== Troubleshooting ==

{| class="wikitable"
! Symptom !! Cause and fix
|-
| No event arrives after a test || Check the test severity first: Info alerts are never sent. Then confirm the appliance can reach the Datadog API host over HTTPS, directly or through its configured proxy.
|-
| Datadog returns 403 || The API key usually belongs to a different Datadog site than the URL host. Use the API host for the site your organization is on.
|-
| Datadog rejects the key || Make sure you used an API key, not an Application key.
|-
| An alert cleared in QuantaStor still shows in Datadog || Expected. QuantaStor sends nothing when an alert is cleared, and Datadog events cannot be closed.
|-
| Error and Critical alerts look the same in Datadog || Datadog has no Critical status. Filter on the {{Code|1=severity}} tag instead.
|}

To see what QuantaStor sent, check the service log at {{Code|1=/var/log/qs/qs_service.log}} on the appliance; the API key appears there masked.

== Related pages ==

* [[Storage System Alert Manager]]
* [[Call-home / Alerting]]
* [[Zabbix Integration]]
* [[PagerDuty Integration]]
* [[Slack Integration]]
* [[QuantaStor CLI Command Reference]]

----
<small>''Verified against QuantaStor 7.0.0.''</small>

[[Category:integration_guide]]

Datadog Integration - Revision history

Qadmin: osn-seo-utilities: datadog-integration @ 7b79d754f6a9 (approved in the portal)