Differences

This shows you the differences between two versions of the page.

--- app:auto_fingerprint [2017/11/29 17:56] – vivek
+++ app:auto_fingerprint [2017/11/29 22:58] (current) – [Web Server Access Log] veera
@@ Line 2: / Line 2: @@
-TLS Fingerprinting is still in its early days therefore the coverage of known prints is not too deep.  The Trisul [[app:tlsfingerprint|TLS Fingerprint App]] ships with a known fingerprint database of about 500 entries ([[https://github.com/trisulnsm/trisul-scripts/blob/master/lua/frontend_scripts/reassembly/ja3/prints/ja3fingerprint.json|ja3prints.json]]) but you will likely find anywhere from 50-70% unknown fingerprints. We need a way to resolve prints to client ID.
+TLS Fingerprinting is still in its early days therefore the coverage of known prints is not too deep.  The Trisul [[app:tlsfingerprint|TLS Fingerprint App]] ships with a known fingerprint database of about 500 entries ([[https://github.com/trisulnsm/trisul-scripts/blob/master/lua/frontend_scripts/reassembly/ja3/prints/ja3fingerprint.json|ja3prints.json]]) but you might find anywhere from 50-70% unknown fingerprints in your network. We need a way to resolve prints to client ID and build up the mappings.
@@ Line 10: / Line 10: @@
 Some of the techniques of resolving unknown fingerprints
-  - Look at the ''User-Agent'' field in the Web Server Access log (if you have access to one)
+  - If you have access to a Web Server log  - Look at the ''User-Agent'' field
-  - See if you can find a ''User-Agent'' vertex nearby, for example if the client followed a 302 Redirect to a https site.
+  - If not, see if you can find a ''User-Agent'' vertex nearby, for example if the client followed a HTTP 302 Redirect to a https site.
-  - Look at Hosts using the print, see if you can detect a pattern or an application. Many web scanners, bots, and applications like Git, Dropbox can be detected this way.
+  - Look at Hosts using the fingerprint, see if you can detect a pattern or an application. Many web scanners, bots, and applications like Git, Dropbox can be detected this way.
@@ Line 19: / Line 19: @@
-Internally we use a Ruby TRP script that can automate this process if given access to web server logs. The script is available at [[https://github.com/trisulnsm/trisul-scripts/tree/master/lua/frontend_scripts/reassembly/ja3/prints|mk_ja3print.rb]]
+Internally we use a Ruby TRP script that can automate this process if given access to web server logs. The script is available on Github at [[https://github.com/trisulnsm/trisul-scripts/tree/master/lua/frontend_scripts/reassembly/ja3/prints|mk_ja3print.rb]]
@@ Line 35: / Line 35: @@
-{"desc":"RSiteAuditor","ja3_hash":"35c0a31c481927f022a3b530255ac080","ja3_str":"771,49192-159-158-157-156-49195-49187-49191-49172-49171-61-60-53-47-49196-49188-49162-49161-106-64-56-50-10-19-5-4,65281-0-10-11-13-35,23-24,0"}
+Running the script.
-{"desc":"urlgrabber/3.10 yum/3.4.3","ja3_hash":"37f691b063c10372135db21579643bf1","ja3_str":"771,49196-49162-49195-52393-49161-49200-49172-49199-52392-49171-159-57-56-107-158-52394-51-50-103-22-19-157-53-61-156-47-60-10-5-4,0-65281-10-11-13,29-23-24-25,0"}
-{"desc":"Feedly/1.0","ja3_hash":"f22bdd57e3a52de86cda40da2d84e83b","ja3_str":"771,49188-49192-61-49190-49194-107-106-49162-49172-53-49157-49167-57-56-49187-49191-60-49189-49193-103-64-49161-49171-47-49156-49166-51-50-49196-49195-49200-157-49198-49202-159-163-49199-156-49197-49201-158-162-49160-49170-10-49155-49165-22-19-255,10-11-13-0,23-24-25-9-10-11-12-13-14-22,0"}
+//Usage : mk_ja3fingerprint.rb  TRP-Server-Endpoint  Webserver-IP  Webserver-Access-Logs  Trisul-TLSPrint-Log-Pattern//
+A sample run of the script is shown below
+<code>
+$ ruby mk_ja3fingerprint.rb  tcp://74.207.234.90:12006  138.68.45.27    'trisul_access.log*'  'lua.stdout.jahash.lua.11*'
-vivek@viveku14:~/bldart/s01/trisul-scripts/lua/frontend_scripts/reassembly/ja3/prints$ ruby mk_ja3fingerprint.rb  tcp://74.207.234.90:12006  138.68.45.27    'trisul_access.log*'  'lua.stdout.jahash.lua.11*'
 "Found 29 Unresolved JA3 TLS Prints"
 "Sending EdgeGraph request vertex key=35c0a31c481927f022a3b530255ac080"
@@ Line 54: / Line 58: @@
 "Sending EdgeGraph request vertex key=1543a7c46633acf71e8401baccbd0568"
 "Sending EdgeGraph request vertex key=f22bdd57e3a52de86cda40da2d84e83b"
+..
+"Output written to file /tmp/fingerprint.json"
+</code>
+Once the script is finished, the JSON output is written to ''/tmp/prints.json'' this can be easily appended to the TLS Prints database.
+<code json>
+{"desc":"RSiteAuditor","ja3_hash":"35c0a31c481927f022a3b530255ac080","ja3_str":"771,49192-159-158-157-156-49195-49187-49191-49172-49171-61-60-53-47-49196-49188-49162-49161-106-64-56-50-10-19-5-4,65281-0-10-11-13-35,23-24,0"}
+{"desc":"urlgrabber/3.10 yum/3.4.3","ja3_hash":"37f691b063c10372135db21579643bf1","ja3_str":"771,49196-49162-49195-52393-49161-49200-49172-49199-52392-49171-159-57-56-107-158-52394-51-50-103-22-19-157-53-61-156-47-60-10-5-4,0-65281-10-11-13,29-23-24-25,0"}
+{"desc":"Feedly/1.0","ja3_hash":"f22bdd57e3a52de86cda40da2d84e83b","ja3_str":"771,49188-49192-61-49190-49194-107-106-49162-49172-53-49157-49167-57-56-49187-49191-60-49189-49193-103-64-49161-49171-47-49156-49166-51-50-49196-49195-49200-157-49198-49202-159-163-49199-156-49197-49201-158-162-49160-49170-10-49155-49165-22-19-255,10-11-13-0,23-24-25-9-10-11-12-13-14-22,0"}
+</code>
+Iteratively running this script for a few days can resolve most of the unknown prints. That makes outlier detection much easier.
+===== Other methods to resolve =====
+Once you get the unknown prints down to 10-20% you can use Trisul's excellent Graph Analytics manually to explore and nail down each print.  We will see that in another article.