One of my customers uses PagerDuty to manage alerts and incidents for the application we are building. We were struggling to integration Azure alerts with the new orchestration functionality in PagerDuty until we learned of the currently undocumented API we should be using.
Emitting Azure alerts into PagerDuty
Wiring up Azure with PagerDuty so that triggered Azure alerts generates an alert the information appears in PagerDuty looks pretty simple on the face of it: You create an Aure Integration which gives you a url including an integration key:
https://events.pagerduty.com/integration/<integration key>/enqueue
You stick that url in your Acton Group as a webhook and away you go - every time an alert fires, data is sent to PagerDuty and you get a json object that you can manipulate.
Our application is pretty complex though, and the support team wanted to build out a service map and route alerts accordingly. Using the Azure Integration this is actually pretty complex - it looked like they would need to define several services in PagerDuty and create a separate azure integration (each with its own integration key) for each one.
There is a better way of doing this in PagerDuty now - orchestrations allow you to create processing rules for alerts as they come into the system. The problem is that this uses the new V2 PagerDuty API and all the docs say you need to send the Integration Key as a header. Action Groups don’t allow you to customise the request - only to specifiy the url so no joy.
Yes, we could create a logic app, as suggested in documentation - the action group calls the logic app url and then we process the alert message, enriching it with the data we need and calling PagerDuty. But this adds another service that we must manage and maintain - and that could break. It’s not a great solution.
The new v2 webhook url
Our friends at PagerDuty were very helpful, once we’d explained our problem. It turns out that there is a url we can use, that includes the integration key just like the v1 API does:
https://events.pagerduty.com/x-ere/<integration key>