Save YouTube videos using Trisul scripting

If you are involved in forensics investigations, you have to pivot quite a bit between data types such as traffic, flows, and packets. You could certainly carry this out via the user interface, but it is extremely tedious to do this repeatedly. This is a problem crying out for automation. The Trisul Remote Protocol is an API that lets you do this kind of thing.

We have a new Github repo to which we will be adding analysis scripts for public use. The directory youtube has scripts that deal with this blog post.

The task

 Save all YouTube videos seen by a Trisul sensor in the last 24 hours as ready to play FLV or WEBM files. Also make sure the filenames are the video titles. 

The steps

Here is how we approach the task.

1. Search for all HTTP URL resources from Trisul containing the string videoplayback. We notice that the request to get the actual video has this string in the URL.
2. Pivot to packets. For each matching URL get a PCAP of the HTTP flows.

At this point we have the PCAP of the video – the tougher task is to get the matching title.

1. Extract using Unsniff the FLV or WEBM file from the PCAP.
2. Search for the HTTP Referer value in the video flow.
3. Search for HTTP URL resources containing the Referer
4. Pivot to packets. Get the HTML Page that refered the video.
5. Extract the HTML file that refered the video.
6. Search for the HTML <title> containing the video name
7. Rename the FLV/WEBM file with the title

Tools

Trisul runs online and is therefore constrained by how much it can do. The idea is the online tool does just enough to index various datatypes. The heavy protocol level lifting is left to offline tools.

One of the key steps in the chain shown above is Extract. This refers to the ability to extract content files out of PCAP streams. We use Trisul’s sister product Unsniff Network Analyzer for this purpose. Like Trisul, Unsniff also has a Ruby scripting API. You can download Unsniff for Windows for free.

How to run ?

We have two scripts on our Github page corresponding to the steps shown above.

youtube_vids.rb	TRP script	Search for videos and download PCAPs
youtube_titles.rb	TRP + Unsniff script	Does the whole thing. Including naming the video titles. To run this you need to be on a Windows box

1. Download the script you want to run.
2. Download the demo certificate and key from the TRP Samples page
3. Make sure you have Ruby installed on your machine
4. Run as shown below – substitute the IP address of the Trisul sensor and the TRP port

Copyruby youtube_titles.rb 192.168.1.22 12001 

The scripts are well commented. Also check out the following

1. TRP API reference
2. Unsniff API reference

Download Trisul 2.6 for Ubuntu or CentOS today.