During one of my “Meet the Expert” sessions this year in Barcelona I was asked if there is an easy way to use SNMP traps as Aria Operations Notification and let the SNMP trap receiver decide what to do with the trap based on included information except for the alert definition, object type or object name itself.
The requirement is to make it as simple and as generic as possible thus creating separate alert definitions and notifications for e.g. Windows and Linux teams or Dev or Test environments is not an option.
Solution
I had few ideas in my mind but I had to test it first as working with SNMP traps is not something I am doing very often.
Basically we have two easy options to include additional information in the notification:
- Add metrics and/or properties to the payload template which can be used as a differentiator.
- Modify the alert definition to always include an additional symptom which can be used as the differentiator, like for example include a vSphere tag based symptom.
Aria Operations Payload Templates allow you to add any additional metrics and properties to the notification. Theses metrics and properties do not have to be related to the actual alert definition but might help to organize and route the alerts in the receiving system based on that additional information.
In the following picture you can see my payload template which includes one additional metric and one property. My test alert definition will be triggered on Virtual Machine object type.
For my tests I have also created an new very simple Symptom Definition, this symptom is basically trigger everytime a Virtual Machine has any vSphere tag assigned to it. A specific tag can be now used to be parsed later on and allow required decisions.
Next picture shows the symptom definition.
My Aria Operations Alert Definition includes the actual symptom I am interested in, for simplicity reasons it is also a certain vSphere tag which I can quickly set and remove to trigger the alert, combined using a boolean AND
with the dummy symptom definition.
As last step in Aria Operations I have created a Notification which will send the SNMP trap to my Aria Orchestrator instance where I can inspect the SNMP message to see what is actually included.
SNMP Message
And here is what Aria Operations is sending as the SNMP message. For completeness I have included the entire message here and highlighted the additional information, both, the dummy symptom and the modified payload. The following links describe the Aria Operations MIB and help identitiv and parse the relevant parts.
=============
oid: 1.3.6.1.2.1.1.3.0
type: Number
snmp type: Timeticks
value: 3112273537
Element 2:
=============
oid: 1.3.6.1.6.3.1.1.4.1.0
type: String
snmp type: OID
value: 1.3.6.1.4.1.6876.4.50.1.0.46
Element 3:
=============
oid: 1.3.6.1.4.1.6876.4.50.1.2.1.0
type: String
snmp type: Octet String
value: vrops.cpod-cmbu-vcf01.az-muc.cloud-garage.net
Element 4:
=============
oid: 1.3.6.1.4.1.6876.4.50.1.2.2.0
type: String
snmp type: Octet String
value: ansible
Element 5:
=============
oid: 1.3.6.1.4.1.6876.4.50.1.2.3.0
type: String
snmp type: Octet String
value: General
Element 6:
=============
oid: 1.3.6.1.4.1.6876.4.50.1.2.4.0
type: String
snmp type: Octet String
value: 1669559583519
Element 7:
=============
oid: 1.3.6.1.4.1.6876.4.50.1.2.5.0
type: String
snmp type: Octet String
value: warning
Element 8:
=============
oid: 1.3.6.1.4.1.6876.4.50.1.2.6.0
type: String
snmp type: Octet String
value: New alert by id 1ab40eba-c480-4475-91e2-a0cc682fe945 is generated at Sun Nov 27 14:33:03 UTC 2022;
Element 9:
=============
oid: 1.3.6.1.4.1.6876.4.50.1.2.7.0
type: String
snmp type: Octet String
value: https://172.28.4.33/ui/index.action#environment/object-browser/hierarchy/d83f38d5-ec7d-44e8-81dc-54b02b3cd3ee/alerts-and-symptoms/alerts/1ab40eba-c480-4475-91e2-a0cc682fe945
Element 10:
=============
oid: 1.3.6.1.4.1.6876.4.50.1.2.8.0
type: String
snmp type: Octet String
value: 1ab40eba-c480-4475-91e2-a0cc682fe945
Element 11:
=============
oid: 1.3.6.1.4.1.6876.4.50.1.2.9.0
type: String
snmp type: Octet String
value: symptomSet: 1242e208-cc7f-40db-9bb0-ecc8a55b1f9b
relation: self
totalObjects: 1
violatingObjects: 1
symptom: tk-Include-vSphere-Tags
active: true
obj.1.name: ansible
obj.1.id: d83f38d5-ec7d-44e8-81dc-54b02b3cd3ee
obj.1.metric:
obj.1.info: Property [<OS-Type-Windows3.11>] matches regular expression .*
symptom: tk-TriggerTestAlert
active: true
obj.1.name: ansible
obj.1.id: d83f38d5-ec7d-44e8-81dc-54b02b3cd3ee
obj.1.metric:
obj.1.info: Property [<OS-Type-Windows3.11>, <killSwitch-On>] contains <killSwitch-On>
Element 12:
=============
oid: 1.3.6.1.4.1.6876.4.50.1.2.10.0
type: String
snmp type: Octet String
value: Application
Element 13:
=============
oid: 1.3.6.1.4.1.6876.4.50.1.2.11.0
type: String
snmp type: Octet String
value: Performance
Element 14:
=============
oid: 1.3.6.1.4.1.6876.4.50.1.2.12.0
type: String
snmp type: Octet String
value: warning
Element 15:
=============
oid: 1.3.6.1.4.1.6876.4.50.1.2.13.0
type: String
snmp type: Octet String
value: warning
Element 16:
=============
oid: 1.3.6.1.4.1.6876.4.50.1.2.14.0
type: String
snmp type: Octet String
value: info
Element 17:
=============
oid: 1.3.6.1.4.1.6876.4.50.1.2.15.0
type: String
snmp type: Octet String
value:
Element 18:
=============
oid: 1.3.6.1.4.1.6876.4.50.1.2.16.0
type: String
snmp type: Octet String
value: VirtualMachine
Element 19:
=============
oid: 1.3.6.1.4.1.6876.4.50.1.2.17.0
type: String
snmp type: Octet String
value: tk-TestAlert-01
Element 20:
=============
oid: 1.3.6.1.4.1.6876.4.50.1.2.18.0
type: String
snmp type: Octet String
value:
Element 21:
=============
oid: 1.3.6.1.4.1.6876.4.50.1.2.19.0
type: String
snmp type: Octet String
value: health
Element 22:
=============
oid: 1.3.6.1.4.1.6876.4.50.1.2.20.0
type: String
snmp type: Octet String
value: tk-SNMP-Trap-Test
Element 23:
=============
oid: 1.3.6.1.4.1.6876.4.50.1.2.21.0
type: String
snmp type: Octet String
value: Number of KPIs Breached : 0.0
Parent Host : esx03.cpod-cmbu-vcf01.az-muc.cloud-garage.net
Element 24:
=============
oid: 1.3.6.1.4.1.6876.4.50.1.2.22.0
type: String
snmp type: Octet String
value:
Element 25:
=============
oid: 1.3.6.1.4.1.6876.4.50.1.2.23.0
type: String
snmp type: Octet String
value:
The following links explain the SNMP content in detail.
https://github.com/librenms/librenms/blob/master/mibs/vmware/VMWARE-VCOPS-EVENT-MIB
https://mibs.observium.org/mib/VMWARE-VROPS-MIB/
Stay safe.
Thomas – https://twitter.com/ThomasKopton