Sweeping traffic with Trisul and OpenIOC

OpenIOC is an XML schema that allows specification and sharing of threat information. An example of OpenIOC is the NetTraveler indicator file 469aed6f-941c-4a1e-b471-3a3e80cbcc2e.ioc

Trisul Remote Protocol lets you write scripts in Ruby to automate Trisul tasks. In this post, we introduce a little ruby script iocsweep.rb which will consume an OpenIOC file, extract supported network based indicators, and sweep past traffic for matches.

How it works

First we use a bit of Nokogiri magic to extract the following indicators

1. PortItem/RemoteIP
2. Network/DNS
3. Network/String
4. FileItem/Md5sum
5. Network/URI

Trisul can also sweep for TLS Certificate Authorities, Cert DER Hashes, Cert Public Key, Network files, IDS alerts, Net Blocks etc. They arent used in the NetTraveler IOC however so we left them out for now.

Use XPath + Nokogiri

If you wanted to get the IPs of all PortItem/remoteIP elements

<IndicatorItem id="f0aafc8a-7551-ca0b-9ada-b73807bf5aae" condition="is">
 <Context document="PortItem" search="PortItem/remoteIP" type="mir" />
 <Content type="IP">98.143.145.80</Content>
</IndicatorItem>

You would use the following XPath technique

# easy to extract the IPs contained inside the indicator..
doc.xpath("//xmlns:IndicatorItem/xmlns:Context[@search='PortItem/remoteIP']")
   .collect do |a|
      a.parent.at_xpath("xmlns:Content").text
   end

Use TrisulRP ruby gem to scan these indicators

Once you have the indicators you can use the TRP Methods to search for these indicators. The iocsweep.rb sample on the GitHub repo trisul-scripts has all the details.

Sweep time

Some indicators such as DNS/URL/IP Blocks are fast to check, others are slower. For example to check FileItem/MD5 against a list Trisul has to reassemble, uncompress, and normalize all HTTP traffic. This could take a while depending on the hardware available. Here are some tips.

1. Treat sweeping for IOCs as a batch job.
2. Could take an hour to scan 1 days of traffic.
3. Searching for strings runs at about 200-300Mbps on our Dual Core Atom appliance. So have processing power ready.
4. Sweep one day at a time so you can view streaming results.
5. You can sweep for multiple IOCs simultaneously.

Sample run with a false positive

We ran the iocsweep.rb script using the NetTraveler IOC on our office network.

This is output we got.

[dhinesh@trp]$ ruby iocsweep.rb 192.168.1.22 12001 469aed6f-941c-4a1e-b471-3a3e80cbcc2e.ioc 
--------------------+-----------
Indicator            Count      
--------------------+-----------
PortItem/remoteIP    17 items 
Network/DNS          39 items 
Network/URI          9 items 
Network/String       5 items 
FileItem/Md5sum      106 items 
--------------------+-----------
Enter PEM pass phrase:

Sweeping for IPs...stand by
Its clean

Sweeping for domains...stand by
We are clean on domains

Sweeping for url content...stand by
All good on HTTP URLs 

Sweeping for [Army Cyber Security Policy 2013.doc]. 
Get a beverage, its going to be a while..
Found 1 matches
Flow 225:19198  \x0A        <Content type="string">Army Cyber Security Policy 2013.doc</Content>\x0D\x0A      </IndicatorIte 

Sweeping for [Report - Asia Defense Spending Boom.doc]. 
Get a beverage, its going to be a while..
Found 1 matches
Flow 225:19198  \x0A        <Content type="string">Report - Asia Defense Spending Boom.doc</Content>\x0D\x0A      </IndicatorIte 

Sweeping for [His Holiness the Dalai Lama???s visit to Switzerland Day 3.doc ]
Get a beverage, its going to be a while..

Found 1 matches
Flow 225:19198  \x0A        <Content type="string">His Holiness the Dalai Lama???s visit to Switzerland Day 3.doc </Content>\x0D\x0A      </IndicatorIte >


Checking all files after reassembly for MD5 match 
Get lunch. Could take a while
Whew! All files MD5 are clean, also check your endpoints

[dhinesh@trp]$

We appear to be all clean on NetTraveler. The “Network/String” matches on the IOCs “His Holiness the Dalai Lama” etc are false hits because we uploaded and viewed the IOC itself which contained these terms. The network simply caught that activity.

Looking for testers

If there are others who are interested in this approach, please contact us We are looking for Beta sites big on security and working with indicators.