ContributorHeartbeats Monitoring - suggestion
Hello,
One of our customer is using a lot of Custom Scripts, where we are monitoring their output by syslog.
There is starting a problem, when script stuck or block. Then customer don't know about it. Is there any possibilities how to monitor, If these scripts are working?
We have suggestion, that Custom Scripts can send also message like OK or KO. 2 Events will match these both. There should be an action (run book) for script is running like:
em7_snippets.generate_alert(xtype = 1, xid=EM7_VALUES['%x'], message = 'script heartbeat')
and not running with count like:
if EM7_VALUES['%c'] >= 2:
em7_snippets.generate_alert(xtype = 1, xid=EM7_VALUES['%x'], message = 'script is not running')
There will be also 2 automation. For script heartbeat will be action for heartbeat every 5min (for example as schedules) and for some Device group.
Second automation for not running will have action not running.
I think, there will must be next two events which will be matched by "script heartbeat or script is not running". Also if new syslog message will come, the first events (mentioned above) will must clear these events for continue with actions and automatons.
Do you think that it is possible to implement it? How will look Hardware usage when we will have more automations/actions or when we will have it on lot of devices? Do you think that it is construable?
Thank you for your answers.
