SIE Remote Access (AXA) Tutorials and other Guides¶
- Farsight Network Message (NMSG)
Tutorial 1: Watch Newly Observed Domains¶
This tutorial assumes you've signed up to receive Farsight's Newly Observed Domains (NOD) datafeed to watch newly active domains and ensure your users don't visit any newly minted – often malicious – domains. It shows you how to examine the Newly Observed Domains (NOD) feed in real time. Commands and their output are listed with discussion below.
1 $ sratool
2 > connect ssh:sra-service@sra-eft.sie-remote.net
3 * HELLO srad version 0.2.5 sra-eft AXA protocol 1
4 > 1 watch ch=211
5 1 OK WATCH started
6 > count 5
7 > channel 211 on
8 * OK CHANNEL ON/OFF channel ch211 on
9 1 ch211 SIE newdomain
10 flyinghorse-colorado.com/A: flyinghorse-colorado.com
11 1 ch211 SIE newdomain
12 treatmentforboils.com/NS: treatmentforboils.com
13 1 ch211 SIE newdomain
14 servicedeck.com/NS: servicedeck.com
15 1 ch211 SIE newdomain
16 www.markenmacher.eu/A: markenmacher.eu
17 1 ch211 SIE newdomain
18 recruitniks.com/NS: recruitniks.com
19 packet count limit exceeded
20 > count
21 packet printing stopped by count 1990 packets ago
Lines 1-3: Invoke sratool, and use the connect command to establish a connection to the SRA server. The connection is managed via SSH, meaning all of the benefits conferred by the SSH protocol are available to sratool. Upon success, the client emits the hello string from the AXA_P_OP_HELLO message which was sent by the server and contains the server's software version, name, and AXA protocol verison.
Lines 4-5: Inform the server we want to watch SIE channel 211 traffic (this is the NOD channel). The server responds with the current watch status. The watch is the most fundamental sratool command. This is how sratool "signs up" to receive data from the SRA server. As its name implies, watch sets up a watch which is a low-level primitive that tells the SRA server that the client is interested in nmsg messages or IP packets that meet one of the following criteria:
- is to, from, or contains the specified address
- contains the specified domain name
- arrived on the specified SIE channel
- are SIE messages that could not be decoded
A watch is given a tag that is an integer label used to refer to the watch. An SRA server connection or session can have zero or more watches at a time and the user can add or delete watches as needed. Note that sratool allows only a single SRA connection at a time.
Line 6: Using the count command, we inform sratool we only want to see 5 packets. After this number is met, sratool will stop emitting packets to the screen (though traffic may still be flowing from server to client).
Lines 7-8: With the channel command, enable channel 211 (NOD). The current channel status is printed. Another fundamental command to sratool is channel. Issued alone on the command-line, it will emit the entire list of available SIE channels for which the user is provisioned.
Lines 9-19: sratool emits 5 NOD packets as it receives then from the server. Once the packet count limit is reached, emission stops.
Lines 20-21: Issuing the count command with no arguments prints the current count status. In this case, we find 1990 NOD packets have been streamed to the client, but since we exceeded our limit, they were not emitted to the screen.
Tutorial 2: Counts And Limits¶
Continuing in the session above, let's tweak a few knobs and press a few buttons.
22 > list watches
23 1 ch=ch211
24 > 1 delete
25 1 OK STOP watch deleted
26 > rate
27 RATE LIMITS
28 unlimited per second; current value=307
29 10 seconds between reports
30 > rate 1
31 RATE LIMITS
32 1 per second; current value=2
33 10 seconds between reports
Lines 22-23: The list watches command prints all of the active watches. We've still got one going, we're just not emitting any packets to the screen.
Lines 24-25: We delete the watch by referencing its tag with the delete command.
Lines 26-29: Another handy command, rate allows us to query the rate limiter and control it. Currently, there is no rate limiting in play – packets will be emitted as quickly as they appear. For lower bandwidth channels, like NOD, this is might not be a problem. For the DNSDB channels, which are much higher bandwidth, we'll want to limit the rate at which those packets are sent by the server to sratool.
Lines 30-33: Using the rate command, we set a rate limit of 1 packet per second. This will come in handy in the last part of the tutorial where we'll examine DNSDB.
Tutorial 3: Watch For A Specific Domain In Farsight's Passive Dns Feed¶
As a bonus, let's peek at SIE channel 202 traffic, Farsight's raw passive DNS feed.
34 > 2 watch dns=*.github.com
35 2 OK WATCH started
36 > channel 202 on
37 * OK CHANNEL ON/OFF channel ch202 on
38 2 ch202 base dnsqr response UDP_QUERY_RESPONSE
39 204.13.250.16.53 > 68.105.29.142.17296 IP TTL=58 UDP 86 bytes
40 DNS: raw.github.com IN A qr aa NOERROR 1 ans, 0 auth, 0 add RRs
41 2 ch202 base dnsqr response UDP_QUERY_RESPONSE
42 208.78.71.16.53 > 208.106.17.39.64372 IP TTL=56 UDP 153 bytes
43 DNS: api.github.com IN A qr aa cd NOERROR 1 ans, 4 auth, 0 add RRs
44 2 ch202 base dnsqr response UDP_QUERY_RESPONSE
45 204.13.250.16.53 > 68.105.28.174.52707 IP TTL=58 UDP 89 bytes
46 DNS: malsup.github.com IN A qr aa NOERROR 1 ans, 0 auth, 0 add RRs
47 * MISSED
48 lost 0 input packets, dropped 0 for congestion,
49 121 for per sec limit
50 since 2014/12/08 17:29:38
51 2 ch202 base dnsqr response UDP_QUERY_RESPONSE
52 204.13.250.16.53 > 68.105.28.174.47116 IP TTL=58 UDP 149 bytes
53 DNS: github.com IN A qr aa NOERROR 1 ans, 4 auth, 0 add RRs
Lines 34-35: We set another watch, this time we want to watch for the wild card domain "*.github.com". Anything matching this domain will be emitted, such as www.github.com and github.com itself.
Lines 36-37: We turn on channel 202, Farsight's raw passive DNS channel.
Lines 38-53: All domains matching the watch are emitted.
Sratunnel¶
At the end of our last tutorial, you probably wondered aloud: "…this is cute but I need real-time bulk transfer of SIE data back to my network. Does this technology even exist in the modern world?"
Yes, yes it does.
sratunnel is the workhorse of the AXA family. It is used to transfer SIE data from the remote server to the local network. It is what Farsight uses for production deployment of SIE data to the customer. sratunnel can be thought of as a fast, efficient, and smart conduit for SIE data. Data goes in one end and sratunnel has a variety of nozzles the user can custom fit on the other end to emit the data into different output formats, including:
- NMSGs to a UDP port
- NMSGs to a TCP port
- NMSGs to a file
- pcap to a file
- pcap to a network interface
Tunnel Newly Observed Domains¶
Now that you know how to use the Newly Observed Domains (NOD) datafeed to watch newly active domains and ensure your users don't visit any newly minted – often malicious – domains, we are going to show you how to to tunnel the data to your local network for bulk analysis.
NMSG Primer¶
To consume the data in this tutorial, you'll need another Farsight implement called nmsgtool. It is a deeply useful all-purpose tool for working with NMSGs (network messages). NMSG is the format Farsight uses to type, structure and package arbirtary data for transit. Much of Farsight's data is packaged and delivered as NMSG. A detailed discussion of the NMSG suite will be covered in future blog series. For now it's just important to understand that you can work with NMSGs using nmsgtool. Onward!
This tutorial will show you, gallant Farsight datafeed customer, how to plumb the Newly Observed Domains (NOD) feed from SIE to your local network, in real time. Commands and their output are listed with discussion below.
1 $ sratunnel -s 'ssh:sra-service@sra-eft.sie-remote.net' \
2 > -c 211 \
3 > -w ch=211 \
4 > -o nmsg:udp:127.0.0.1,8430
Line 1: Invoke sratunnel. The -s option instructs the tool where and how to connect. The option string should look familiar, it's the same one used with sratool with the same intent and results. Securely connect via SSH as sra-service to sra-eft.sie-remote.net.
Line 2: The -c option sets the channel you want to stream. We want NOD which is channel 211.
Line 3: The -w option sets the watch. You learned last week that a watch is how to inform the tool what to look for. In this case, everything on channel 211.
Line 4: Finally, we specify -o, which tells sratunnel where to put the data it streams. In the case above, we've snapped on a shiney new "NMSGs to localhost on port 8430" nozzle and that's where we'll find our output. Well done! You've plumbed your first SRA session. Data is aflowin'. Let's build a small corpus and have a look…
5 $ nmsgtool -l 127.0.0.1/8430 \
6 > -c 20000 \
7 > -o channel-211.txt
8 $ head -8 channel-211.txt
9 [98] [2014-12-16 23:31:06.438992023] [2:5 SIE newdomain] [a1ba02cf] [] []
10 domain: befrenshee.com.
11 time_seen: 2014-12-16 23:28:19
12 rrname: befrenshee.com.
13 rrclass: IN (1)
14 rrtype: NS (2)
15 rdata: ns67.domaincontrol.com.
16 rdata: ns68.domaincontrol.com.
17 $ grep ^domain: channel-211.txt | awk '{print $2}' > NOD.txt
Line 5: We use nmsgtool to connect to the loopback address on port 8430.
Line 6: The -c option specifies a maximum count of payloads to capture.
Line 7: The -o option tells nmsgtool to write presentation output to a file.
Line 8: Let's examine one entry…
Line 9: Each NMSG datagram contains a fixed-length header containing the message size, a UTC timestamp, the message type, a 32-bit source identifier and optional SIE operator and group codes (both empty in this case).
Line 10: The fresh young domain, hot off the press!
Line 11: How fresh? At the time of this writing, that timestamp of when the domain was observed was just over two minutes old.
Lines 12-16: The DNS meta-data associated with the domain.
Line 17: You're free to manipulate the data however you see fit. As per the above, you can take the list of 20,000 young domains and feed the file into the young domain crunching automation of your choice… :)