Counter Group Monitor

BACKEND SCRIPT

Monitor counter group activity.

Get called when metrics related to a counter group are computed and stored.

Structure

Table cg_monitor

The table consists the following

counter_guid String or function returning string Counter Group to attach to for monitoring
onbeginflush engine, timestamp Before starting to flush all metrics to db
onflush engine, timestamp, key, arrayofmetrics Called for each key as they are being flushed
onendflush engine After all keys have been flushed for this interval
onbegintopperflush engine, timestamp, meter Before flushing toppers for this meter
ontopperflush engine, key, metric Called for each topper item
onendtopperflush engine, meter After topper flush
onupdate engine, timestamp, key, arrayofmetrics As each update happens (1sec resolution)
onnewkey engine, timestamp, key A new key was discovered within the stream window.
onmetronome engine, timestamp, tick_count, tick_interval called every second ( Tick Interval)

Functions Reference

Function onbeginflush

Purpose

Before a counter group is flushed to the Trisul database on the Hub node. Trisul is a streaming analytics system. By default every 60 seconds the analytics are snapshotted and sent to the database node (hub). The onbeginflush function is therefore called every 60 seconds.

When called

When an flush operation is about to start. The sequence goes



-- operation is invoked every 60 seconds by default 
--
onbeginflush(..) 
onflush(..)
onflush(..)
onflush(..)
..
..
onendflush(..)

Parameters

engine An engine object use this object to add metrics, counter items, or counter items into the Trisul framework
timestamp Timestamp Timestamps seconds tv_sec

Return value

Ignored

Example

Function onflush

Purpose

Custom processing before each counter item is flushed. Perhaps write to your own tools or logfiles.

When called

Just before each counter item is flushed to the database. The maximum delay between getting a onnewcounter item and a corresponding onflush(..) for that counter item is 60 seconds.

Parameters

engine An engine object use this object to add metrics, counter items, or counter items into the Trisul framework
timestamp Timestamp Timestamps seconds tv_sec
key string the key identifying the counter item
arrayofmetrics array of numbers array of metrics. array item 0 refers to meter 0 and so forth

Return value

Ignored

Example


Function onendflush

Purpose

The flush operation has ended. You can do some cleanup operations here.

When called

When all the counter items in this timeslice have been flushed to the database.

Parameters

engine An engine object use this object to add metrics, counter items, or counter items into the Trisul framework
timestamp Timestamp Timestamps seconds tv_sec

Return value

Ignored

Example


Function onbegintopperflush

Purpose

Topper snapshotting is a key streaming analytics step in Trisul. Just as raw metrics are tracked using the onbeginflush, onflush, and onendflush functions documented above, topper snapshot flushes can be handled by you using the onbegintopperflush, ontopperflush, onendtopperflush functions.

When called

By default every 60 seconds. The call flow goes



-- flushing Topper for metric 0 
onbegintopperflush(engine,148858585,0) -- flushing topper meter 0 
ontopperflush(..)
ontopperflush(..)
ontopperflush(..)
..
onendtopperflush(,0)

-- flushing Topper for metric 1 
onbegintopperflush(engine,148858585,1) -- flushing topper meter 1
ontopperflush(..)
ontopperflush(..)
ontopperflush(..)
..
onendtopperflush(..)


Parameters

engine An engine object use this object to add metrics, counter items, or counter items into the Trisul framework
timestamp Timestamp Timestamps seconds tv_sec
meter number The meter number representing the topper tracker set. For example : when “Top Hosts By Connections” are flushed – the meter will be 6 where 6 represents the metric “Connections”

Return value

Ignored

Example


Function ontopperflush

Purpose

Each topper item as it is flushed. You can leverage Trisul’s highly tuned streaming computation of toppers.

When called

When each topper item is flushed.

Parameters

engine An engine object use this object to add metrics, counter items, or counter items into the Trisul framework
key string The topper item
metric number The metric for the key and the meter. Remember the meter is sent in onbegintopperflush

Return value

Ignored

Example


Function onendtopperflush

Purpose

The flush operation has ended. You can do some cleanup operations here.

When called

When all the topper items for a particular metric have been flushed

Parameters

engine An engine object use this object to add metrics, counter items, or counter items into the Trisul framework
meter number The meter number

Return value

Ignored

Example


Function onupdate

High frequency function
For busy networks this can result in thousands of updates every second. Keep your LUA function onupdate(..) efficient and avoid I/O or blocking.

Purpose

A real time update to a metric item.

When called

A streaming update to a counter item. This will update a metric value in real time in 1 second resolution. Therefore for busy networks you can expect many keys to be updated every second in the common case.

Parameters

engine An engine object engine use ths to add your results back into the Trisul framework
timestamp Timestamp Timestamps seconds tv_sec
key string the key identifying the counter item
arrayofmetrics array of numbers array of metrics. array item 0 refers to meter 0 and so forth

Return value

Ignored

Example


Function onnewkey

Purpose

A new key was discovered. Handle when you see keys you havent seen before (in the recent past)

When called

When “new” keys are seen by Trisul in this counter group. Trisul uses a Bloom Filter per counter group to track newly seen keys, the filter itself is reset when it becomes saturated. Use this method to track newly seen keys, but keep in mind they are not “absolutely” new as seen by Trisul since it first started.

Parameters

engine An engine object engine use ths to add your results back into the Trisul framework
timestamp Timestamp Timestamps seconds tv_sec
key string the key identifying the counter item

Return value

Ignored

Example


Function onmetronome

Purpose

Plug into a metronome.

When called

If you define a onmetronome(..) function you will be plugged into the Trisul metronome heartbeat mechanism. This method will be called every metronome tick(roughly every second). The context in which this method is called is threadsafe and you can add metrics to the Engine from here.

Parameters

engine An Engine object use this object to add metrics, resources, or alerts into the Trisul framework
timestamp Number Current timestamp (tv_sec epoch seconds)
tick_count Number An incremeting tick counter
tick_interval Number The tick interval, in seconds.

Return value

Ignored

Example