One of the many exciting things about Geocortex Analytics is its ability to run a custom batch script when an alarm is triggered. This can be useful for performing actions like restarting important resources, and relaying alarm messages to a third-party messaging platform.
To demonstrate this, we wanted to focus today’s Geocortex Tech Tip on how powerful this can be. In the video below, we’ll show you how to run a batch script from an Analytics alarm to a Slack channel.
“Hi, my name is Stephen, and I am from the Analytics team. Today, we are going to look at how to execute a batch script when an alarm is triggered. Let’s take a look!
So first of all, I’m going to show you how to set up a batch script that will relay triggered Analytics alarm data to a Slack channel, and it does this by setting in an alarm message to a Slack webhook endpoint whenever an alarm is triggered.
Before I go and show you how to configure all this, let’s take a look at the end result. Here I have a Slack channel — #alarms, I’ve called it — and you can see all the messages here are just triggered alarms that come from Analytics. For example, we have an HTTP response time alarm, that is triggered on a geocode service called ‘Charlotte,’ and it tells you all that information as well as providing the URL, if you want to click on that and take a quick look into the resource.
This could be really useful if your organization uses Slack, especially if you receive desktop notifications, so that you are notified immediately as soon as your resources are down or unavailable or is experiencing performance difficulties.
Now I’m going to show you how to configure this, so you can have these messages show up in your own Slack channel.
I’m going to go to the server, which I’ve installed Analytics and on here I have a batch file that is responsible for sending that data to the Slack end point.
So here is the batch file, and as you can see it’s using ‘curl’ to send that data to the endpoint. We’ve set the content type as json and according to the Slack API specifications, we have a text key here and the value is the message that you want to show up. You will also notice that we have these kind of placeholder command line parameters. When Analytics triggers an alarm and there’s a batch file associated with that alarm, when it runs that batch file, it passes in command line arguments that you can then optionally utilize. In this case, ‘%~5’ is the alarm type, ‘%~2’ is the display name of the resource and ‘%~4’ is the URL of that resource if it exists. And you can consort the documentation to get a list of all of these command line parameters.
For now you can see that I just configured a pretty short sentence as well as that, you need to know the URL of course for the Slack end point. To get that, you need to go into Slack, you need to create an app, and you need to activate incoming webhooks. Once you do so, you’ll be provided with a URL and a very similar looking sample curl request to use to post to that channel.
You can just copy that across, and then once you have your own batch file, what you need to do is you need to copy that batch file specifically to the install directory and then “/Data/Analytics.ResourceCollecter/Scripts.” Once the batch file is in here, Analytics will be able to detect it, and you’ll be able to select it when you are going to an Analytics configuration.
Let me go back to Analytics and here I am. Here you would go into any resource that you want to configure that batch file on. In other words, whatever resources you would want to be notified about in Slack when an alarm triggers related to them.
For instance, here I am going to go into an ‘ArcGIS Servers,’ and these are the default settings that apply to all ArcGIS servers, not just one. I’ll go to ‘Alarms’ right now; I could look at the response time alarm for all my ArcGIS Servers and here I have ‘slack.bat’ selected. You can see any of the batch files that I have in that directory were also displayed here, but I could also go into another one, where I don’t have it currently selected. Hit save, and now whenever a CPU usage alarm is triggered on any ArcGIS servers, I’m going to be notified in my Slack channel when that occurs. That’s how you can set up Analytics to automatically relay triggered alarm information into a Slack channel.
Next, I’m going to briefly demonstrate how to restart an ArcGIS server when we detect it is unavailable or down.
For that, I’m going to go back to my remote server and so the approach we are going to go with for this, is it we want to restart the actual ArcGIS server process on the ArcGIS Server. To do this, it would basically be the equivalent of going to the task manager on that machine and stopping the ArcGIS Server service and then starting up again and hoping that that will resolve whatever issues are currently occurring in the environment.
This is a bit of an inelegant approach to actually fixing ArcGIS Server, but because we’re trying to do something remotely, this is our best effort for trying to resolve the issue.
What I’ve done here, is I’ve already downloaded a batch file that is responsible for restarting the service, and so you would just provide the machine name and the service that you wish to restart. The way that this batch file works is that it first stops the service, I can see sc – stop, waits until that service has stopped and then starts the service again by sc – start.
The reason you need a proper batch file with all this extra logic in, is that sc – stop and start or asynchronous operations and you really need to wait until the service is either stopped or started before you then perform the next action.
So, this isn’t the actual batch file that I’m going to reference from Analytics; instead I have this other one, ‘restart_service.bat,’ and this is calling my other batch file along with the parameters for the server name, as well as the service name that I wish to restart, which is just ‘ArcGIS Server’ in this case. So I’ve created both of these batch files, I’ve copied them to the ‘Scripts’ directory, it’s really only important to copy this file to the Scripts directory because that’s what I’m going to link to, the other one can be anywhere as long as you reference it correctly from this file.
Now I’m going to go back into Analytics and actually link that batch file to the ArcGIS Server. So, I’m going to go to the ArcGIS Server in question, this 1071 ArcGIS Server. I’ll just go into ‘Alarms,’ and in this case, I only really care about setting the batch file on the request failure alarm, I’m not bothered if it takes a while to respond or if it is using a lot of CPU memory. I’m going to edit this, and as you can see, I’ve already set this up. So, I’ve overridden the default settings, and then I selected this batch file, and then hit ‘Save.’
Whenever that alarm is next triggered, what’s going to happen is that the batch file will be executed and we’re going to try to restart that ArcGIS Server process remotely and hopefully that’s going to fix any lingering issues.
That concludes this Tech Tip. I hope you’ve learned how useful it is to run a batch script when an alarm triggers in Analytics.
Thanks for watching and happy scripting!”
Want to learn more about how Geocortex Analytics can help you optimize your performance? Click the button below for more information about the product, or to schedule a demo!