Object Engine

Functions

A summary of the functions available in this object.

• F – functions that can be called from Frontend scripts
• B – functions that can be called from Backend scripts

If you call functions that are not available you will get the following error

ERR_NOTAVAIL: add_flow_counter() is not available for backend scripts

Function update_counter

Adds a single metric to the Trisul platform.

Purpose

Update a counter with a value. Recall that the heirarchy in Trisul is

• Counter Group example Applications
  • Key example pop3
    • Meter example meter id = 5 Active Connections

Also see

Use update_counter_raw if

1. you are updating GAUGE or AVERAGE meter data types
2. you are updating more than 8 metrics for the same key

Parameters

Return value

None

Usage

..

-- update the Hosts counter group identified by the GUID {4CD..}
--        the Host identified by 182.28.22.123
--        the Meter 1 (received bytes)
--           with the value 288278
--
engine:update_counter( "{4CD742B1-C1CA-4708-BE78-0FCA2EB01A87}", "182.28.22.123", 1, 288378)
..

Function update_counter_bytes

FRONTEND ONLY

Adds a single metric to the Trisul framework. Same as update_counter but where the metric value is the packet wire length.

Purpose

When used with simplecounter and other front end scripts – this method adds a metric with the wire length of the packet context in which it is called.

Parameters

Return value

None

Usage

This method is only available on frontend scripts.

Function update_counter_raw

A variation of update_counter

Purpose

This does exactly the same thing as update_counter with the following difference.

1. this version has a slightly lower performance
2. use this method if you want to update a large number of keys for a single counter group . More than 8 keys
3. you are updating GAUGE or AVERAGE meter data types

Parameters

Return value

None

Usage

--
-- we are updating 6 keys for a single counter group
-- for this kind of updating use the 'raw' version
--
engine:update_counter_raw("{D2AAD7C6-...}", "user_1",  1, 100)
engine:update_counter_raw("{D2AAD7C6-...}", "user_11", 1, 10)
engine:update_counter_raw("{D2AAD7C6-...}", "user_2",  1, 200)
engine:update_counter_raw("{D2AAD7C6-...}", "user_33", 1, 3200)
engine:update_counter_raw("{D2AAD7C6-...}", "user_34", 1, 300)
engine:update_counter_raw("{D2AAD7C6-...}", "user_44", 1, 23200)

Function update_key_info

Set a user friendly label for a key.

Purpose

Set a user friendly label or a description for a particular key. For example : to assign a hostname to an IP address

Parameters

Return value

None

Usage

..
--
-- to assign a label "Router1_10G_1" to the MAC address "0A:11:21.BA:00:02" 
-- the GUID "{4B0.."  represents the MAC Counter Group
-- 
engine:update_key_info( "{4B09BD22-3B99-40FC-8215-94A430EA0A35}", "0A:11:21:BA:00:02", "Router1_10G_1")
..

Function add_alert

Add an alert to the Trisul framework.

Purpose

Lets you add a simple alert to the Trisul pipeline. This simple version only contains a priority, alert-type, and description. Also see add_alert_full and add_alert_tca for more specific variants.

Parameters

Return value

None

Usage

The following example adds a Priority 1 alert to the general purpose User Alerts group.

engine:add_alert("{B5F1DECB-51D5-4395-B71B-6FA730B772D9}", -- GUID for 'User Alerts' group
                 resource:flow():id(),                     -- flow ID
                 "RAMNIT-DETECTED",                        -- alert key , think of this as a SigID   
                 1,                                        -- priority
                 "A potential EK was detected message ="   -- free form text alert 
                 )

Function add_alert_full

Add an alert to the Trisul framework with all bells and whistles.

Purpose

Same as add_alert but with classification and priority fields. To make it compatible with alerts from IDS systems like Snort and Suricata

Parameters

Return value

None

Usage

Function add_alert_tca

A special type of alert called Threshold Crossing Alert (TCA) based on metrics.

Purpose

Same as add_alert family but designed for metrics based alerting.

Parameters

Return value

None

Usage

Function add_resource

Add a new resource.

Purpose

Resources are meta data which you can log and search on. You can create your own resource group or add to existing resource groups like “DNS Records”, “HTTP URLs”, etc. Think of resources as logs and resource groups as log types

Parameters

Return value

None

Usage

-- add a custom SHA256 hash resource
--
engine:add_resource('{9781db2c-f78a-4f7f-a7e8-2b1a9a7be71a}',  -- represents FileHash resource in Trisul 
                    flowkey:id(),                              -- flow ID
                    "SHA256:"..hex_sha256,                     -- resource key (log summary)
                    full_filename)                             -- full resource  (full log)

Function timestamp

The latest timestamp in seconds since Epoch format (timeval.tv_sec) seen by the engine.

Purpose

If you want to find out the latest timestamps seen by the engine. Remember that you may not be able to simply use os.date) because Trisul also works on PCAP files that may have other timestamps.

Parameters

None.

Return value

1. For frontend scripts that work with packets, this represents the timestamp of the last packet seen.
2. For backend scripts, this returns the last stream interval that was processed to completion.

Usage

Function add_flow_counter

FRONTEND ONLY

Add an automatic flow based counter.

Purpose

A Flow Counter works in the following way :

1. you attach a counter to a flow
2. all packets in that flow from that point onwards automatically get that counter with the metric equal to the packet length

Parameters

Return value

None

Usage

The method is only available for Frontend scripts. If you call this method from a backend script you will get an error.

If you detect a particular flow to be carrying YouTube video you can attach a counter “you-tube-traffic” metric. From that point onwards each packet on that flow will automatically update the ‘youtube-traffic’ counter with the packet length.

Note on the Direction parameter

Use Direction=0 to update a Total traffic counter and Directions 1,2 to update IN/OUT separately. In the snippet below you’ve created your own counter group with GUID {55F..} with three meters 0=Total,1=Xmit,2=Recv. You can see how to update the Totals counter Direction=0 and In/Out counters

-- Direction 0 : update both OUT and IN direction packets to the same counter

-- to update a TOTAL BYTES counter 
engine:add_flow_counter( flowkey:id(),
                         "{55F98CC1-646E-4DE6-AD79-6B896211F177}",  -- a counter guid
                         'my-video-traffic', 
                         0,  -- Total bytes
                         0)  -- Both directions update the Total bytes counter 


-- for XMIT 
engine:add_flow_counter( flowkey:id(),
                         "{55F98CC1-646E-4DE6-AD79-6B896211F177}",  -- a counter guid
                         external_host, 
                         1,   -- Transmit meter
                         1)   -- Only OUT direction update the Trasmit meter 
-- for RECV
engine:add_flow_counter( flowkey:id(),
                         "{55F98CC1-646E-4DE6-AD79-6B896211F177}",  -- a counter guid
                         external_host, 
                         2,   -- Recv meter 
                         2)   -- Only IN direction update the receive meter 

Function reset_flow_counter

FRONTEND ONLY

Removes a flow based counter.

Purpose

Remove a previous flow based counter added by add_flow_counter

Parameters

Return value

None

Usage

The method is only available for Frontend scripts. If you call this method from a backend script you will get an error.

Function tag_flow

Tag a flow with a string.

Purpose

Add a text label to a IP flow. You can then search for flows using this string.

Parameters

Return value

None

Usage

Function instanceid

A string representing the instance of the engine

Purpose

There may be multiple engine contexts in Trisul. This string identifies each one. You can use this to construct unique filenames etc.

Parameters

None.

Return value

Usage

A common use of instanceid is to construct filenames for log files. Another use is when you have scheduled scripts and you only want to run them once. In that case, you only run your code when engine:instanceid()=="0" See engine_monitor for more.

Function probeid

The probe id.

Purpose

Use this to retrieve the probeid – eg probe0 that the engine object belongs to.

Parameters

None.

Return value

Usage

Function update_flow

FRONTEND ONLY

Add a flow metric.

Purpose

Update a flow based metric. The key used in a SessKey format.

Parameters

Return value

None

Usage

Function new_flow_record

Use this method only when you are writing an Input Filter plugin importing feeds with “Netflow like” flow records.

Purpose

Tell the Trisul framework we have new flow record with X bytes and Y packets. The reason you need to do this is because some plugins ; such as Badfellas work with flow records for efficiency reasons. This method does not impact metrics in any way, but exists only to inform some plugins of a new flow.

Parameters

Return value

None

Usage

Function set_flow_duration

FRONTEND ONLY

Set length in seconds of a flow.

Purpose

Use this along with update_flow to handle Netflow like input data. This method sets the length of each flow. Use this if the duration of the flow is available in your input records.

Parameters

Return value

None

Usage

Function disable_reassembly

FRONTEND ONLY

Stop reassembly and reconstruction of this flow.

Purpose

This method is useful in various contexts like file extraction and reassembly

The main use of this method is when you need to see the beginning bytes of a TCP flow to determine whether you are interested in further reassembly. Turning off reassembly of TCP streams you are not interested in can save significant CPU and memory.

Parameters

Return value

None

Usage

disable_reassembly is only available for frontend scripts that work with packets and flows.

Function post_message_frontend

Post a message to the frontend (packet) pipeline

Purpose

Use this method to communicate with other LUA plugins that are plugged into the packet pipeline stages.

You can use trisulctl_probe testbench guid to generate a msg id.

Parameters

Return value

None

Usage

-- sent to all plugins which have an onmessage(..) handler 
--
engine:post_message_frontend("192.168.2.1 port 299 needs fileextraction")


-- sent to only those plugins who have subscribed to 
-- message if {F8D5570E-00A7-41FD-C30E-298C55D8EC4D}
--
engine:post_message_frontend("{F8D5570E-00A7-41FD-C30E-298C55D8EC4D}",
                              "192.168.2.1 port 299 needs fileextraction")

..

Function post_message_backend

Post a message to the backend (streaming metrics) pipeline

Purpose

Use this method to communicate with other LUA plugins that are plugged into the backend pipeline

You can use trisulctl_probe testbench guid to generate a msg id.

Parameters

Return value

None

Usage

-- sent to all backend plugins which have an onmessage(..) handler 
--
engine:post_message_backend("192.168.2.1 port 299 label all flows as suspicious")


-- sent to only those plugins who have subscribed to 
-- message id {F8D5570E-00A7-41FD-C30E-298C55D8EC4D}
--
engine:post_message_backend("{F8D5570E-00A7-41FD-C30E-298C55D8EC4D}",
                              "192.168.2.1 port 299 label all flows as suspicious")

..

Function add_edge

Add a new uni-directional edge to the edge stream. This is part of the new Trisul Streaming Graph Analytics feature set.

Purpose

In Graph Analytics lingo, an edge is two vertices joined with a connection. The add_edge method allows to you push a simple Edge to the Trisul streaming pipeline.

The above picture explains the following terms

Subject Vertex Type

The type or color of Vertex-A. This is a GUID , typically a counter group or alert type.

Subject Vertex Key

A key identifying the vertex within the group. Example an IP Address within Hosts

Target Vertex Type

The type or color of Vertex-B.

Target Vertex Key

A key identifying the target vertex

Parameters

Note on edge direction.

Trisul edges are directional, so if you want to add a bi-directional edge. Call add_edge twice with A and B reversed the second time. See Usage section.

Return value

None

Usage

Here is how you add a bi-directional edge. Say between a PING-ALERT and an Internal IP address

--
engine:add_edge("{F8D5570E-00A7-41FD-C30E-298C55D8EC4D}", "ICMP-PING-ALERT"
                "{98348488-45A7-45FD-450E-45838053800D}", "10.2.28.18" );

-- the other direction
engine:add_edge("{98348488-45A7-45FD-450E-45838053800D}", "10.2.28.18",
                "{F8D5570E-00A7-41FD-C30E-298C55D8EC4D}", "ICMP-PING-ALERT");

                              

..

Function add_flow_edges

Add edges from a given vertex to all vertices of a flow.

Purpose

This function adds the following uni-directional edges from a given vertex

• Vertex → Source IP
• Source IP → Vertex
• Vertex → Dest IP
• Dest IP → Vertex
• Port → Vertex
• Vertex → Port
• Protocol → Vertex
• Vertex → Protocol

Parameters

Return value

None

Usage

Add an ALERT vertex to all the endpoints of the flow that generated it.

--
-- adds edges from the new ICMP-PING-ALERT vertex
--
engine:add_flow_edges("06A:C0.A8.01.02:p-0B94_D1.D8.F9.3A:p-0016", 
                      "{F8D5570E-00A7-41FD-C30E-298C55D8EC4D}", "ICMP-PING-ALERT");

..

Function set_key_attribute

Add an arbitrary attribute-name, attribute-value pair to any key.
If the key already has an attribute with the attribute-name then overwrite the old value with the new attribute_value

Purpose

Allows you to attach a name,value to any key. For example

1. A common use is to assign an Interface Speed (ifSpeed) to a netflow interface.
2. other uses can be thought of such as “operating-system” to a Host IP, etc

Parameters

Return value

None

Usage

--
-- to assign a value of 100000 to interface_speed for interface 1ge/0/0
--

set_key_attribute(  "{7CA09636-02D4-45e7-AA00-BE0D49B94E26}", -- netflow interfaces group
                    "1ge/0/0",             -- interface key 
                    "interface_speed",     -- attr_name
                    "1000000" )            -- value

Function add_flow_attribute

Adds an attribute-name=attribute-value to a flow.

Purpose

Flow attributes can be attached to any flow. You can then write other scripts that can listen to these attributes using reassembly/onattribute and work on them.

Parameters

Return value

None

Usage