Network Security Monitoring aka NSM involves the collection of a variety of information elements by passive observation of network traffic. There are essentially two ways to run packet analytics, Live Capture vs Reading PCAP files. They may appear to be the same, but there are some crucial differences that are hidden only to surface when you try to read in PCAP files.
When you have a large PCAP file , say covering 6 hours of traffic. You have two choices, you can choose to replay the file to a NSM system using a tool like TCPReplay. But if you did that at normal rate it would take 6 hours !! If you speed it up or play it at top speed, you lose crucial information about traffic metrics.
But if you played it back at natural rate, then that is an enormous waste of CPU and time. You could be processing packets at the natural rate of 1MB/s for hours when your powerful CPU can do 800Mbps. This is why we prefer to read PCAP files, presumably we can just let our CPU rip through as fast as possible.
All packet capture tools I know are clocked by the timestamps present in the packets themselves.
In a Live traffic capture, the wall time is the clock. During low traffic periods your CPU and Memory usage goes down, but the time moves at fixed rate. Tick, tock..
When you read PCAPs, the time moves faster or slower depending on how fast your tool can inject the PCAP files.
A process-A that generates info-A can rip through the PCAP file at 100Mbps, another process-B generating info-B may only be able to process at 5Mbps. If both of these information streams hit a single backend, then we may have these problems.
A lot of this depends on the capabilities of the database.
Some protocols need to maintain state. A good example is FTP which has a control channel which sets up a data channel. Another is SSL Session resumption, or SIP/TRP for VOIP, etc.
Your tools needs to maintain this state in order to decode the protocols correctly. A single threaded processing pipeline may not need a state synchronization but a multi threaded pipe does.
Even for PCAP file reads, we can benefit by multiple threads. Trisul uses 2 threads by default, you can also use 4, or 8 threads to speed up PCAP file reads. The problem here is the following.
There are some other less important differences, we can document in the next article.