Save YouTube videos using Trisul scripting
If you are involved in forensics investigations, you have to pivot quite a bit between data types such as traffic, flows, and packets. You could certainly carry this out via the user interface, but it is extremely tedious to do this repeatedly. This is a problem crying out for automation. The Trisul Remote Protocol is an API that lets you do this kind of thing.
We have a new Github repo to which we will be adding analysis scripts for public use. The directory youtube has scripts that deal with this blog post.
Save all YouTube videos seen by a Trisul sensor in the last 24 hours as ready to play FLV or WEBM files. Also make sure the filenames are the video titles.
Here is how we approach the task.
- Search for all HTTP URL resources from Trisul containing the string
videoplayback. We notice that the request to get the actual video has this string in the URL.
- Pivot to packets. For each matching URL get a PCAP of the HTTP flows.
At this point we have the PCAP of the video – the tougher task is to get the matching title.
- Extract using Unsniff the FLV or WEBM file from the PCAP.
- Search for the
HTTP Referervalue in the video flow.
- Search for HTTP URL resources containing the Referer
- Pivot to packets. Get the HTML Page that refered the video.
- Extract the HTML file that refered the video.
- Search for the HTML
<title>containing the video name
- Rename the FLV/WEBM file with the title
Trisul runs online and is therefore constrained by how much it can do. The idea is the online tool does just enough to index various datatypes. The heavy protocol level lifting is left to offline tools.
One of the key steps in the chain shown above is Extract. This refers to the ability to extract content files out of PCAP streams. We use Trisul’s sister product Unsniff Network Analyzer for this purpose. Like Trisul, Unsniff also has a Ruby scripting API. You can download Unsniff for Windows for free.
How to run ?
We have two scripts on our Github page corresponding to the steps shown above.
|youtube_vids.rb||TRP script||Search for videos and download PCAPs|
|youtube_titles.rb||TRP + Unsniff script||Does the whole thing. Including naming the video titles. To run this you need to be on a Windows box|
- Download the script you want to run.
- Download the demo certificate and key from the TRP Samples page
- Make sure you have Ruby installed on your machine
- Run as shown below – substitute the IP address of the Trisul sensor and the TRP port
Copyruby youtube_titles.rb 192.168.1.22 12001
The scripts are well commented. Also check out the following
Download Trisul 2.6 for Ubuntu or CentOS today.