OSNEXUS Online Documentation Site

PagerTree Integration

This page shows how to send QuantaStor alerts to PagerTree. QuantaStor posts each Warning, Error and Critical alert to a PagerTree Webhook (Custom) integration, which sets the PagerTree urgency from the alert severity and pages whoever the integration's Destinations name. When you close the alert in QuantaStor, PagerTree resolves it.

The setup has three steps in the Alert Manager and PagerTree, followed by reference sections:

Step 1: Create a Webhook (Custom) integration in PagerTree

QuantaStor talks to a PagerTree Webhook (Custom) integration. You do not need a PagerTree user API key or a team id: the integration's Destinations decide who is paged.

  1. In PagerTree, open Integrations and click Create Integration.
  2. Choose Webhook (Custom).
  3. Enter a Name, for example quantastor-alerthandler.
  4. Leave Enabled checked.
  5. Set Destinations to the team (or teams) that should be paged for storage alerts.
  6. Leave Urgency at its default. QuantaStor sends an urgency with every alert, and the definition below applies it.
  7. Paste the YAML Definition below into the YAML Definition field.
  8. Click Create.
  9. Open the new integration and copy its Endpoint URL. It has the form https://api.pagertree.com/integration/int_aAbBcCdDeEfF01.

Treat the Endpoint URL as a secret. Anyone who has it can open alerts in your PagerTree account without any other credential.

YAML Definition

Paste this definition exactly as shown. The first rule resolves an alert when QuantaStor closes it; the second creates the PagerTree alert and fills in its title, description, urgency, tags and Additional Data rows. 

rules:
  - match:
      log.data.event_type:
        $eq: "resolve"
    actions:
      - type: resolve
        thirdparty_id: "{{log.data.Id}}"
  - match:
      log.data.event_type:
        $eq: "create"
    actions:
      - type: create
        thirdparty_id: "{{log.data.Id}}"
        title: "{{log.data.Title}}"
        description: "{{log.data.Description}}"
        urgency: "{{log.data.Urgency}}"
        tags: "{{log.data.Tags}}"
        additional_data:
          - format: text
            label: Severity
            value: "{{log.data.Meta.severity}}"
          - format: text
            label: Storage System
            value: "{{log.data.Meta.storage_system}}"
          - format: text
            label: Storage System ID
            value: "{{log.data.Meta.storage_system_id}}"
          - format: text
            label: Storage Grid
            value: "{{log.data.Meta.storage_grid}}"
          - format: text
            label: Alert Type
            value: "{{log.data.Meta.alert_type}}"
          - format: text
            label: Alert Time
            value: "{{log.data.Meta.alert_time}}"
          - format: text
            label: Service Version
            value: "{{log.data.Meta.service_version}}"
          - format: text
            label: Kernel Version
            value: "{{log.data.Meta.kernel_version}}"
          - format: text
            label: Serial Number
            value: "{{log.data.Meta.serial_number}}"
          - format: text
            label: System Startup
            value: "{{log.data.Meta.system_startup}}"
          - format: text
            label: Alert ID
            value: "{{log.data.Meta.alert_id}}"

Step 2: Add the PagerTree endpoint in QuantaStor

Navigation: Storage Management → Storage System (toolbar group) → Alert Manager (toolbar) → ITSM Integrations (tab)
 
The ITSM Integrations tab of the Alert Manager with a PagerTree endpoint added. Choose PagerTree in the Module drop-down, paste the Endpoint URL into Webhook URL, then click Add.

The ITSM Integrations tab of the Alert Manager lists every IT service management endpoint the grid sends alerts to, one row per module and URL.

  1. Open the Alert Manager and select the ITSM Integrations tab.
  2. In Module, select PagerTree.
  3. Paste the PagerTree Endpoint URL into Webhook URL as it is. Do not append anything to it.
  4. Click Add. The endpoint appears in the list.
  5. Click OK or Apply to save.
 
The Module drop-down lists every ITSM module the appliance supports, PagerTree among them.

To stop sending alerts to PagerTree, select the row, click Remove Selected, then OK or Apply.

QuantaStor sends alerts through the appliance's configured HTTPS proxy, if there is one, so an appliance without direct Internet access can still reach PagerTree.

From the CLI

Add the endpoint with qs alert-config-set, giving the module name and the URL separated by a colon: 

qs alert-config-set --endpoints="pagertree:https://api.pagertree.com/integration/int_aAbBcCdDeEfF01"

To remove it, prefix the entry with ~: 

qs alert-config-set --endpoints="~pagertree:https://api.pagertree.com/integration/int_aAbBcCdDeEfF01"

Step 3: Send a test alert

 
Send User Generated Alert, opened from Generate Test Alert. Severity offers Warning, Error and Critical.

Click Generate Test Alert at the bottom of the Alert Manager. The Send User Generated Alert dialog opens with two fields:

  • Message -- the alert text.
  • Severity -- Warning (the default), Error or Critical.

Click OK. A new alert appears in PagerTree, routed to the integration's Destinations.

From the CLI, use qs alert-raise. Always set --alert-severity: the command defaults to info, and QuantaStor never sends Info alerts to ITSM modules, so a test at the default level reaches PagerTree with nothing. 

qs alert-raise --title="PagerTree test" --message="Testing the PagerTree integration" --alert-severity=error

Severity and urgency

QuantaStor maps each alert's severity to a PagerTree urgency:

QuantaStor severity PagerTree urgency
Warning medium
Error high
Critical critical
Info not sent

Info alerts are never sent to PagerTree, or to any other ITSM module. If an integration looks broken, check the severity of the alert you tested with first.

Resolving alerts

QuantaStor keys each PagerTree alert as quantastor- followed by the QuantaStor alert id, which PagerTree shows as the Thirdparty ID. When you close the alert in QuantaStor, the handler sends a resolve with the same key and PagerTree resolves the matching alert.

Close an alert with qs alert-clear and --close=true. List alerts and their ids with qs alert-list. 

qs alert-list
qs alert-clear --filter=<alert id> --close=true

To close every open alert at once, use qs alert-clear-all --close=true.

Dismiss Alerts in the web interface removes alerts from QuantaStor without sending a close, so the matching PagerTree alerts stay open. Resolve them in PagerTree, or close the alerts from the CLI instead.

What the PagerTree alert contains

PagerTree field Content
Title The alert title, prefixed with its SNMP alert type id in brackets, for example [<alert type id>] <title>
Description The QuantaStor alert's description
Urgency From the alert severity (see Severity and urgency)
Tags quantastor, severity:<level>, system:<system name>, grid:<grid name>, alert_type_id:<id>
Thirdparty ID quantastor-<alert id>

The YAML Definition also adds eleven Additional Data rows to the alert page: Severity, Storage System, Storage System ID, Storage Grid, Alert Type, Alert Time, Service Version, Kernel Version, Serial Number, System Startup and Alert ID.

Troubleshooting

  • Nothing arrives in PagerTree. Check the test alert's severity first: Info alerts are never sent. Then confirm the Endpoint URL in the ITSM Integrations tab matches the one PagerTree shows, with nothing appended, and that the integration is Enabled.
  • Alerts show as "dropped" in PagerTree. The alert was delivered; nobody acknowledged it. This happens, for example, when the Destinations team has no on-call schedule. Fix the team's schedule in PagerTree.
  • Alerts arrive late. When PagerTree answers with a rate-limit (429) or server (5xx) error, QuantaStor retries the delivery for up to 90 seconds.
  • PagerTree alerts do not resolve. Close the alert with --close=true; dismissing it in the web interface does not send a resolve. Also check that the integration's YAML Definition contains the resolve rule above.

Related pages

Verified against QuantaStor 7.0.0.

Retrieved from "https://wiki.osnexus.com/index.php?title=PagerTree_Integration&oldid=28371"