====== Automatically resolving unknown TLS Fingerprints ======
TLS Fingerprinting is still in its early days therefore the coverage of known prints is not too deep. The Trisul [[app:tlsfingerprint|TLS Fingerprint App]] ships with a known fingerprint database of about 500 entries ([[https://github.com/trisulnsm/trisul-scripts/blob/master/lua/frontend_scripts/reassembly/ja3/prints/ja3fingerprint.json|ja3prints.json]]) but you might find anywhere from 50-70% unknown fingerprints in your network. We need a way to resolve prints to client ID and build up the mappings.
===== Resolving TLS Fingerprints =====
Some of the techniques of resolving unknown fingerprints
- If you have access to a Web Server log - Look at the ''User-Agent'' field
- If not, see if you can find a ''User-Agent'' vertex nearby, for example if the client followed a HTTP 302 Redirect to a https site.
- Look at Hosts using the fingerprint, see if you can detect a pattern or an application. Many web scanners, bots, and applications like Git, Dropbox can be detected this way.
===== Web Server Access Log =====
Internally we use a Ruby TRP script that can automate this process if given access to web server logs. The script is available on Github at [[https://github.com/trisulnsm/trisul-scripts/tree/master/lua/frontend_scripts/reassembly/ja3/prints|mk_ja3print.rb]]
The script isnt too complicated. It works in the following way.
- Connects and gets list of unresolved JA3 TLS Prints in a 24 hour period.
- For Each unresolved print
- Use Graph Analytics to get list of Hosts using it
- Look for the Host in the Web Server Access log and pick out the first User-Agent
- Look for the print in the TLS Print App log - this contains the print and the Print String
- Print it out in JSON format
Running the script.
//Usage : mk_ja3fingerprint.rb TRP-Server-Endpoint Webserver-IP Webserver-Access-Logs Trisul-TLSPrint-Log-Pattern//
A sample run of the script is shown below
$ ruby mk_ja3fingerprint.rb tcp://74.207.234.90:12006 138.68.45.27 'trisul_access.log*' 'lua.stdout.jahash.lua.11*'
"Found 29 Unresolved JA3 TLS Prints"
"Sending EdgeGraph request vertex key=35c0a31c481927f022a3b530255ac080"
"35c0a31c481927f022a3b530255ac080 resolved to Mozilla/5.0 (compatible; RSiteAuditor) 771,49192-159-158-157-156-49195-49187-49191-49172-49171-61-60-53-47-49196-49188-49162-49161-106-64-56-50-10-19-5-4,65281-0-10-11-13-35,23-24,0"
"Sending EdgeGraph request vertex key=37f691b063c10372135db21579643bf1"
"37f691b063c10372135db21579643bf1 resolved to urlgrabber/3.10 yum/3.4.3 771,49196-49162-49195-52393-49161-49200-49172-49199-52392-49171-159-57-56-107-158-52394-51-50-103-22-19-157-53-61-156-47-60-10-5-4,0-65281-10-11-13,29-23-24-25,0"
"Sending EdgeGraph request vertex key=c2769dbd398f0b72e409887ceb9eb8ad"
"Sending EdgeGraph request vertex key=05af1f5ca1b87cc9cc9b25185115607d"
"05af1f5ca1b87cc9cc9b25185115607d resolved to Mozilla/5.0 (Windows NT 6.1; WOW64; rv:40.0) Gecko/20100101 Firefox/40.1 769,49172-49171-57-51-53-47-49162-49161-56-50-10-19-5-4,0-10-11-23-65281,23-24,0"
"Sending EdgeGraph request vertex key=1885aa9927f99ed538ed895d9335995c"
"1885aa9927f99ed538ed895d9335995c resolved to Mozilla/55 (Windows NT 10.0; WOW64; rv:55.0) Gecko/20100101 Firefox/55 771,49195-49199-158-49162-49161-49171-49172-49159-49169-51-50-57-156-47-53-10-5-4-255,0-11-10-35-13-15,14-13-25-11-12-24-9-10-22-23-8-6-7-20-21-4-5-18-19-1-2-3-15-16-17,0-1-2"
"Sending EdgeGraph request vertex key=05e15a226e00230c416a8cdefeb483c7"
"Sending EdgeGraph request vertex key=1543a7c46633acf71e8401baccbd0568"
"Sending EdgeGraph request vertex key=f22bdd57e3a52de86cda40da2d84e83b"
..
"Output written to file /tmp/fingerprint.json"
Once the script is finished, the JSON output is written to ''/tmp/prints.json'' this can be easily appended to the TLS Prints database.
{"desc":"RSiteAuditor","ja3_hash":"35c0a31c481927f022a3b530255ac080","ja3_str":"771,49192-159-158-157-156-49195-49187-49191-49172-49171-61-60-53-47-49196-49188-49162-49161-106-64-56-50-10-19-5-4,65281-0-10-11-13-35,23-24,0"}
{"desc":"urlgrabber/3.10 yum/3.4.3","ja3_hash":"37f691b063c10372135db21579643bf1","ja3_str":"771,49196-49162-49195-52393-49161-49200-49172-49199-52392-49171-159-57-56-107-158-52394-51-50-103-22-19-157-53-61-156-47-60-10-5-4,0-65281-10-11-13,29-23-24-25,0"}
{"desc":"Feedly/1.0","ja3_hash":"f22bdd57e3a52de86cda40da2d84e83b","ja3_str":"771,49188-49192-61-49190-49194-107-106-49162-49172-53-49157-49167-57-56-49187-49191-60-49189-49193-103-64-49161-49171-47-49156-49166-51-50-49196-49195-49200-157-49198-49202-159-163-49199-156-49197-49201-158-162-49160-49170-10-49155-49165-22-19-255,10-11-13-0,23-24-25-9-10-11-12-13-14-22,0"}
Iteratively running this script for a few days can resolve most of the unknown prints. That makes outlier detection much easier.
===== Other methods to resolve =====
Once you get the unknown prints down to 10-20% you can use Trisul's excellent Graph Analytics manually to explore and nail down each print. We will see that in another article.