Skip to main content
Version: 7.15

DSL Processor

Introduction​

The DSL Processor is a powerful tool designed to streamline and enhance your data processing workflows. It allows you to define and execute complex data transformations using a simple and intuitive domain-specific language (DSL). The NetObserv Trap Collector leverages Benthos Bloblang as a DSL and includes additional SNMP-specific functions to enrich and transform the data from SNMP traps.

Features​

  • Intuitive Syntax: Write clear and concise transformation rules.
  • High Performance: Optimized for speed and efficiency.
  • Flexibility: Easily adaptable to various data processing needs.

Custom Functions​

Trapcoll DSL Processor supports the following custom functions:

snmp_display_string()​

The snmp_display_string() transforms a raw OCTET STRING to a DisplayString value (per SNMPv2-TC). In the case of an error while processing the raw value, an empty string is returned.

Example:

root.out.netapp.productSerialNum = this.trap.VarBinds.index(1).Value.snmp_display_string()

snmp_date_and_time()​

The snmp_date_and_time() transforms a raw OCTET STRING to a DateAndTime value (per SNMPv2-TC). In the case of an error processing the raw value an empty string is returned.

Example:

root.out.juniper.jnxOperatingRestartTime = this.trap.VarBinds.index(0).Value.snmp_date_and_time().ts_unix_milli()

snmp_int_enum_enrich() - was enum_enrich()​

The snmp_int_enum_enrich() function transforms an integer to the enumerated string that it represents. Integer enumerations are located by default in /etc/elastiflow/snmp/enums/integer. Note that enum_enrich() is deprecated. While it still works, it will be removed in a future release.

Example:

root.out.IETF.ospfLsdbType = this.trap.VarBinds.index(2).Value.snmp_int_enum_enrich(".1.3.6.1.2.1.14.4.1.2")

Syntax​

snmp_int_enum_enrich("[OID]")

Parameters​

  • OID (string): The Object Identifier used as a key for the enumeration keys.

snmp_mac_address()​

The snmp_mac_address() function transforms a raw OCTET STRING to a MacAddress value (per SNMPv2-TC). In the case of an error processing the raw value an empty string is returned.

Example:

root.out.cisco.cVpcSystemOperMacAddress = this.trap.VarBinds.index(2).Value.snmp_mac_address()

snmp_octet_string()​

The snmp_octet_string() function transforms a raw OCTET STRING to a hex string, e.g. 0708090a0b0c. In the case of an error processing the raw value an empty string is returned.

Example:

root.out.IETF.nlmLogVariableOpaqueVal = this.trap.VarBinds.index(0).Value.snmp_octet_string()

snmp_oid_get_index()​

The snmp_oid_get_index() function extracts the Index portion of a variable binding OID following a supplied prefix.

Example:

root.out.object.index = this.trap.VarBinds.index(0).OID.snmp_oid_get_index(".1.3.6.1.2.1.14.4.1.2")

Syntax​

snmp_oid_get_index("[OID]")

Parameters​

  • OID (string): The Object Identifier used as a key for the enumeration keys.

snmp_oid_extract_index()​

The snmp_oid_extract_index() function extracts the index values from a variable binding OID based on a comma-separated list of index types. The output is an array of index values.

Example:

root.out.object.index.snmp_oid_extract_index("IpAddress,Integer,IpAddress,IpAddress")

Syntax​

snmp_oid_extract_index("[comma-separated list of Index value types]")

Parameters​

  • comma-separated list of Index value types (string): The list of index value types to extract from the OID index.

The following is a list of supported values. Note that these are the same index value types supported by the NetObserv SNMP Collector object definitions.

  • Integer
  • OctetString
  • ImplicitOctetString
  • ObjectIdentifier
  • ImplicitObjectIdentifier
  • Integer32
  • IpAddress
  • MacAddress
  • Unsigned32
  • Opaque

snmp_inet_address()​

The snmp_inet_address() function transforms an InetAddress value (per INET-ADDRESS-MIB) into an IP address, IP address with Zone, or DNS name based on an Integer value of InetAddressType.

Example:

root.out.TEST.inetAddress = this.trap.VarBinds.index(2).Value.snmp_inet_address(root.out.TEMP.inetAddressType)

Syntax​

snmp_inet_address([InetAddressType])

Parameters​

  • InetAddressType (Integer): The value of InetAddressType which indicates the type of the InetAddress that is being transformed.

snmp_int_to_ipv4()​

The snmp_int_to_ipv4() function transforms an Integer raw value into an IPv4 address string.

Example:

root.out.TEST.intToIPv4 = this.trap.VarBinds.index(3).Value.snmp_int_to_ipv4()

Examples of using the custom functions​

Extract and transform index values from the OID of a variable binding​

The following variable binding contains and instance of ospfLsdbType from OSPF-MIB.

{
  Name:  ".1.3.6.1.2.1.14.4.1.2.0.0.0.103.3.155.172.64.15.192.168.215.36", // ospfLsdbType
  Type:  Integer,
  Value: 3,
},

First we extract the index portion of the variable bindings OID:

root.out.snmp.object.index = this.trap.VarBinds.index(0).OID.snmp_oid_get_index(".1.3.6.1.2.1.14.4.1.2")

Which produces the following output:

{
    "snmp":
        "object": {
            "index": "0.0.0.103.3.155.172.64.15.192.168.215.36"
        }
    }
}

Next we extract the individual components of the index, assigning them to a temporary variable that can be deleted later.

root.TEMP.ospfLsdbEntry = root.out.object.index.snmp_oid_extract_index("IpAddress,Integer,IpAddress,IpAddress")

We can then assign the values from the array to fields in the output record. Note the use of snmp_int_enum_enrich() to also transform root.out.ospf.LsdbType to its enumerated string value.

root.out.ospf.LsdbAreaId = root.TEMP.ospfLsdbEntry.index(0)
root.out.ospf.LsdbType = root.TEMP.ospfLsdbEntry.index(1).snmp_int_enum_enrich(".1.3.6.1.2.1.14.4.1.2")
root.out.ospf.LsdbLsid = root.TEMP.ospfLsdbEntry.index(2)
root.out.ospf.LsdbRouterId = root.TEMP.ospfLsdbEntry.index(3)

The resulting output record is now:

{
    "ospf": {
        "LsdbAreaId": "0.0.0.103",
        "LsdbLsid": "155.172.64.15",
        "LsdbRouterId": "192.168.215.36",
        "LsdbType": "summary link"
    },
    "snmp":
        "object": {
            "index": "0.0.0.103.3.155.172.64.15.192.168.215.36"
        }
    }
}

Handling the various forms of IP addresses​

SNMP has various ways of expressing IP addresses.

  • Originally SNMP only supported IPv4 addresses, the IpAddress type.
  • As an IPv4 address is 4 bytes in size, some vendors choose to use a 32-bit integer to express the four bytes.
  • To add support for IPv6 and other possible types, the INET-ADDRESS-MIB introduced InetAddress and InetAddressType, where the latter declares the type, and thus the necessary transformation, of the former. As a result these two value are commonly seen together.

Handling values of types InetAddress and InetAddressType​

Let's look at this last use-case first. Consider the following variable bindings which contain an InetAddressType and an InetAddress.

{
  Name:  ".1.2.3.4.5.6.7.8.9.0.1", // IPv6 InetAddressType
  Type:  Integer,
  Value: 2,
},
{
  Name:  ".1.2.3.4.5.6.7.8.9.0.2", // IPv6 InetAddress
  Type:  OctetString,
  Value: []byte{0x20, 0x1, 0x0d, 0xb8, 0x85, 0xa3, 0x00, 0x00, 0x00, 0x00, 0x8a, 0x2e, 0x03, 0x70, 0x73, 0x34},
},

Let's first assign the InetAddressType to a field.

root.out.MyInetAddressType = this.trap.VarBinds.index(1).Value

We can then pass this InetAddressType value into snmp_inet_address() to transform the InetAddress variable binding's value.

root.out.MyInetAddress = this.trap.VarBinds.index(2).Value.snmp_inet_address(root.out.MyInetAddressType)

The resulting record fields are:

{
    "MyInetAddress": "2001:db8:85a3::8a2e:370:7334",
    "MyInetAddressType": 2
}

Note that we could also use snmp_int_enum_enrich() to further transform MyInetAddressType, or we could drop the field if it is no longer needed after the transformation of MyInetAddress.

Handling IPv4 addresses expressed as a 32-bit integer​

As mentioned above, an IPv4 address is 4 bytes in size, so some vendors will choose to use a 32-bit integer to express the four bytes of the address.

{
  Name:  ".1.2.3.4.5.6.7.8.9.0.3", // 192.168.1.1 as an Unsigned32
  Type:  Uinteger32,
  Value: uint32(3232235777),
},

The snmp_int_to_ipv4() function allows the integer value to be easily transformed into an IPv4 address.

root.out.MyIntToIPv4 = this.trap.VarBinds.index(3).Value.snmp_int_to_ipv4()

The resulting output is:

{
    "MyIntToIPv4": "192.168.1.1"
}