DNSDB for Resilient

Farsight DNSDB integration for IBM Resilient SOAR platform is a collection of functions that connect to DNSDB API. These functions are also accompanied with Resilient workflows that demonstrate the use of each function. These workflows can then be triggered by Resilient rules to enrich the Indicators of Compromise (IoC) with Passive DNS Data provided by DNSDB API. The extension package is bundled with all of these components (functions, workflows, rules).

DNSDB is an artifact enrichment solution. Queries are possible for:

• IP Addresses
• DNS Names

App Host Installation

All the components for running DNSDB function in a container already exist when using the App Host app.

To install,

• Navigate to Administrative Settings and then the Apps tab.
• Click the Install button and select the downloaded file: app-fn_dnsdb-x.x.x.zip.
• Go to the Configuration tab and edit the app.config file and provide the DNSDB API key and Server URL.

[fn_dnsdb]
apikey =
server =

Integration Server Installation

• Resilient platform >= v37.2.49
• An Integration Server running resilient_circuits>=30.0.0
• To set up an Integration Server see: https://ibm.biz/res-int-server-guide
• If using API Keys, minimum required permissions are:
  • Org Data: Read, Edit
  • Function: Read

Installation

• Download the fn_dnsdb.zip.
• Copy the .zip to your Integration Server and SSH into it.
• Unzip the package:

$ unzip fn_dnsdb-x.x.x.zip

• Change Directory into the unzipped directory:

$ cd fn_dnsdb-x.x.x

• Install the package:

$ pip install fn_dnsdb-x.x.x.tar.gz

• Import the configurations into your app.config file:

$ resilient-circuits config -u -l fn-dnsdb

• Import the fn_dnsdb customizations into the Resilient platform:

$ resilient-circuits customize -y -l fn-dnsdb

• Open the config file, scroll to the bottom and edit your fn_dnsdb configurations:

$ nano ~/.resilient/app.config

Config	Required	Example	Description
apikey	Yes	d41d8cd98f00b204e9800998ecf8427e	DNSDB API key
server	Yes	https://api.dnsdb.info	DNSDB API Server

• Save and Close the app.config file.
• [Optional]: Run selftest to test the Integration you configured:

$ resilient-circuits selftest -l fn-dnsdb

• Run resilient-circuits or restart the Service on Windows/Linux:

$ resilient-circuits run

Uninstall the Integration

• SSH into your Integration Server.
• Uninstall the package:

$ pip uninstall fn-dnsdb

• Open the config file, scroll to the [fn_dnsdb] section and remove the section or prefix # to comment out the section.
• Save and Close the app.config file.

Troubleshooting

There are several ways to verify the successful operation of a function.

Resilient Action Status

• When viewing an incident, use the Actions menu to view Action Status.
• By default, pending and errors are displayed.
• Modify the filter for actions to also show Completed actions.
• Clicking on an action displays additional information on the progress made or what error occurred.

Resilient Scripting Log

• A separate log file is available to review scripting errors.
• This is useful when issues occur in the pre-processing or post-processing scripts.
• The default location for this log file is: /var/log/resilient-scripting/resilient-scripting.log.

Resilient Logs

• By default, Resilient logs are retained at /usr/share/co3/logs.
• The client.log may contain additional information regarding the execution of functions.

Resilient-Circuits

• The log is controlled in the .resilient/app.config file under the section [resilient] and the property logdir.
• The default file name is app.log.
• Each function will create progress information.
• Failures will show up as errors and may contain python trace statements.

Support

Name	Version	Author	Support URL
fn_dnsdb	1.0.0	Farsight Security, Inc.	https://farsightsecurity.com

Use Cases

DNSDB Co-Located Hosts

This use case describes the desire to easily identify Hosts that are co-located (based on Address) based on the input of a Host and a given point in time. The response would be a set of domains that also shared the same IP.

• Example Pre-Process Script:

if incident.start_date:
  time_first_before = str(incident.start_date/1000 + 86400)
  time_last_after = str(incident.start_date/1000 - 15552000)
else:
  time_first_before = str(incident.create_date/1000 + 86400)
  time_last_after = str(incident.create_date/1000 - 15552000)


inputs.pivot = """
[
\{\{"function": "rrset", "owner_name": "{0}", "rrtype": "{1}", "limit": {2}, \
    "time_first_before": "{3}", "time_last_after": "{4}"}},
\{\{"function": "rdata", "type": "ip", "pivot": "rdata"}}
]
""".format(artifact.value, "A", 10, time_first_before, time_last_after)

• Example Post-Process Script:

ip_records = workflow.properties.a_records["dnsdb_records"] +
    workflow.properties.aaaa_records["dnsdb_records"]

rrname_list = []

for item in ip_records:
  rrname_list.append(item["rrname"])


c_rdata_list = []

for item in workflow.properties.cname_records["dnsdb_records"]:
  for i in item["rdata"]:
    c_rdata_list.append(i)

aggregate_results = rrname_list + c_rdata_list

hosts_list_set = set(aggregate_results)
msg = ""
for item in hosts_list_set:
  msg += "{}".format(item)
msg += ""

incident.addNote(helper.createRichText("{1} Co-Located Hosts Found for \
    {0}{2}".format(artifact.value, str(len(hosts_list_set)), msg)))

DNDB Historical Address

This use case describes the desire to identify all Addresses used as DNS A and AAAA records for a given Host based on a time window from a starting and stopping point in time.

• Example Pre-Process Script:

inputs.owner_name = artifact.value
inputs.limit = 100
inputs.rrtype = 'A'

#calculate time_first_before, time_last_after based on incident occur date.
if incident.start_date:
  inputs.time_first_before = str(incident.start_date/1000 + 86400)
  inputs.time_last_after = str(incident.start_date/1000 - 15552000)
else:
  inputs.time_first_before = str(incident.create_date/1000 + 86400)
  inputs.time_last_after = str(incident.create_date/1000 - 15552000)

• Example Post-Process Script:

rdata_records = workflow.properties.a_records["dnsdb_records"] +
    workflow.properties.aaaa_records["dnsdb_records"]
rdata_list = []

for item in rdata_records:
  for i in item["rdata"]:
    rdata_list.append(i)

rdata_list_set = set(rdata_list)
msg = ""
for item in rdata_list_set:
  msg += "{}".format(item)
msg += ""

incident.addNote(helper.createRichText("{1} Historical Address Found for \
    {0}{2}".format(artifact.value, str(len(rdata_list_set)), msg)))

DNSDB Historical Hosts

This use case describes the desire to identify all Hosts that resolved to a given Address based on a time window from a starting and stopping point in time.

• Example Pre-Process Script:

inputs.value=artifact.value
inputs.type = 'ip'
inputs.rrtype = 'ANY'
inputs.limit = 100

#calculate time_first_before, time_last_after based on incident occur date.
if incident.start_date:
  inputs.time_first_before = str(incident.start_date/1000 + 86400)
  inputs.time_last_after = str(incident.start_date/1000 - 15552000)
else:
  inputs.time_first_before = str(incident.create_date/1000 + 86400)
  inputs.time_last_after = str(incident.create_date/1000 - 15552000)

• Example Post-Process Script:

dnsdb_records = results["dnsdb_records"]

host_list = []
for item in dnsdb_records:
  host_list.append(item["rrname"])

host_names_msg = ""
for item in set(host_list):
  host_names_msg += "{}".format(item)
host_names_msg += ""

incident.addNote(helper.createRichText("{1} Historical Hosts Found for: \
    {0}{2}".format(artifact.value, str(len(set(host_list))), host_names_msg)))

Function - DNSDB Flex

DNSDB Flex function allows you to search DNSDB by regular expressions and globs (aka wildcarding).

• DNSDB Flex Reference Guide
• Inputs:

Name	Type	Required	Description
key	select	Yes	The search key, can be rrnames(supports “forward” searches based on the owner name of an RRSet) or rdata(supports “inverse” searches based on RData record values).
limit	number	No	Limit for the number of results returned via these lookup methods.
method	select	Yes	The search method, it can be regex(regular expression search) or glob(full wildcarding).
offset	number	No	How many rows to offset (e.g. skip) in the results.
rrtype	text	No	The resource record type of the RRSet, either using the standard DNS type mnemonic, or an RFC 3597 generic type, i.e. the string TYPE immediately followed by the decimal rrtype number.
time_first_after	text	No	Provide results after the defined timestamp for when the DNS record was first observed. For example, the URL parameter “time_first_after=-31536000” will only provide results that were first observed within the last year.
time_first_before	text	No	Provide results before the defined timestamp for when the DNS record was first observed. For example, the URL parameter “time_first_before=1420070400” will only provide matching DNS records that were first observed before (or older than) January 1, 2015.
time_last_after	text	No	Provide results after the defined timestamp for when the DNS record was last observed. For example, the URL parameter “time_last_after=-2678400” will only provide results that were last observed after 31 days ago.
time_last_before	text	No	Provide results before the defined timestamp for when the DNS record was last observed. For example, the URL parameter “time_last_before=1356998400” will only provide results for DNS records that were last observed before 2013.
value	text	Yes	The value to search DNSDB records.

• Outputs:

results = {
        "dnsdb_records": [
            {
                "from_zone_file": false,
                "rrname": "farsightsecurity.yahoo.com.au",
                "rrtype": "CNAME"
            },
            {
                "from_zone_file": false,
                "rrname": "farsightsecurity-2432183.starbucks.com.cn",
                "rrtype": "A"
            },
            {
                "from_zone_file": false,
                "rrname": "farsightsecurity.com.cn",
                "rrtype": "NS"
            },
            {
                "from_zone_file": false,
                "rrname": "farsightsecurity.com.cn",
                "rrtype": "SOA"
            },
            {
                "from_zone_file": false,
                "rrname": "farsightsecurity.com.cn",
                "rrtype": "CNAME"
            },
            {
                "from_zone_file": false,
                "rrname": "www.farsightsecurity.com.cn",
                "rrtype": "CNAME"
            },
            {
                "from_zone_file": false,
                "rrname": "farsightsecurity.damai.cn",
                "rrtype": "CNAME"
            }
        ]
}

Workflows

• Example Pre-Process Script:

inputs.value = artifact.value

• Example Post-Process Script:

dnsdb_records = results['dnsdb_records']

incident.addNote(helper.createRichText("{0} DNSDB Flex Records \
    Found".format(str(len(dnsdb_records)))))

Function - DNSDB RData

This function queries DNSDB’s RData index, which supports “inverse” lookups based on RData record values. In contrast to the RRSet lookup method, RData lookups return only individual resource records and not full resource record sets, and lack bailiwick metadata. An RRSet lookup on the owner name reported via an RData lookup must be performed to retrieve the full RRSet and bailiwick.

• See Farsight DNSDB API Version 2 Documentation section on "rdata lookups".
• Inputs:

Name	Type	Required	Tooltip
aggr	boolean	No	Aggregated results group identical RRSets across all time periods and is the classic behavior from querying the DNSDB.
limit	number	No	Limit for the number of results returned via these lookup methods.
offset	number	No	How many rows to offset (e.g. skip) in the results.
rrtype	text	No	The resource record type of the RRSet, either using the standard DNS type mnemonic, or an RFC 3597 generic type, i.e. the string TYPE immediately followed by the decimal rrtype number.
time_first_after	text	No	Provide results after the defined timestamp for when the DNS record was first observed. For example, the URL parameter “time_first_after=-31536000” will only provide results that were first observed within the last year.
time_first_before	text	No	Provide results before the defined timestamp for when the DNS record was first observed. For example, the URL parameter “time_first_before=1420070400” will only provide matching DNS records that were first observed before (or older than) January 1, 2015.
time_last_after	text	No	Provide results after the defined timestamp for when the DNS record was last observed. For example, the URL parameter “time_last_after=-2678400” will only provide results that were last observed after 31 days ago.
time_last_before	text	No	Provide results before the defined timestamp for when the DNS record was last observed. For example, the URL parameter “time_last_before=1356998400” will only provide results for DNS records that were last observed before 2013.
type	select	Yes	The type specifies how value is interpreted. type may be name, ip or raw
value	text	Yes	The value to search DNSDB records.

• Outputs:

results = {
        "dnsdb_records": [
            {
                "count": 606,
                "from_zone_file": false,
                "rdata": [
                    "www.farsightsecurity.com."
                ],
                "rrname": "scout.dnsdb.info",
                "rrtype": "CNAME",
                "time_first": "2020-03-27T18:37:24Z",
                "time_last": "2020-10-21T18:11:47Z"
            },
            {
                "count": 121,
                "from_zone_file": false,
                "rdata": [
                    "www.farsightsecurity.com."
                ],
                "rrname": "scout-beta.dnsdb.info",
                "rrtype": "CNAME",
                "time_first": "2020-08-20T22:52:29Z",
                "time_last": "2020-10-20T17:54:58Z"
            },
            {
                "count": 546,
                "from_zone_file": false,
                "rdata": [
                    "www.farsightsecurity.com."
                ],
                "rrname": "81.64-26.140.160.66.in-addr.arpa",
                "rrtype": "PTR",
                "time_first": "2013-12-10T01:20:08Z",
                "time_last": "2020-10-10T15:47:19Z"
            }
        ]
}

Workflows

• Example Pre-Process Script:

inputs.value = artifact.value

• Example Post-Process Script:

dnsdb_records = results['dnsdb_records']

incident.addNote(helper.createRichText("{0} DNSDB RData Records \
    Found".format(str(len(dnsdb_records)))))

Function - DNSDB RRSet

This function queries DNSDB’s RRSet index, which supports “forward” lookups based on the owner name of an RRSet.

• See Farsight DNSDB API Version 2 Documentation section on "rrset lookups".
• Inputs:

Name	Type	Required	Tooltip
aggr	boolean	No	Aggregated results group identical RRSets across all time periods and is the classic behavior from querying the DNSDB.
bailiwick	text	No	The “bailiwick” of an RRSet in DNSDB observed via passive DNS replication is the closest enclosing zone delegated to a nameserver which served the RRset. The “bailiwick” of an RRSet in DNSDB observed in a zone file is simply the name of the zone containing the RRSet.
limit	number	No	Limit for the number of results returned via these lookup methods.
offset	number	No	How many rows to offset (e.g. skip) in the results.
owner_name	text	Yes	DNS name specified in DNS presentation format.
rrtype	text	No	The resource record type of the RRSet, either using the standard DNS type mnemonic, or an RFC 3597 generic type, i.e. the string TYPE immediately followed by the decimal rrtype number.
time_first_after	text	No	Provide results after the defined timestamp for when the DNS record was first observed. For example, the URL parameter “time_first_after=-31536000” will only provide results that were first observed within the last year.
time_first_before	text	No	Provide results before the defined timestamp for when the DNS record was first observed. For example, the URL parameter “time_first_before=1420070400” will only provide matching DNS records that were first observed before (or older than) January 1, 2015.
time_last_after	text	No	Provide results after the defined timestamp for when the DNS record was last observed. For example, the URL parameter “time_last_after=-2678400” will only provide results that were last observed after 31 days ago.
time_last_before	text	No	Provide results before the defined timestamp for when the DNS record was last observed. For example, the URL parameter “time_last_before=1356998400” will only provide results for DNS records that were last observed before 2013.

• Outputs:

results = {
        "dnsdb_records": [
            {
                "bailiwick": "com",
                "count": 19,
                "from_zone_file": true,
                "rdata": [
                    "ns.lah1.vix.com.",
                    "ns1.isc-sns.net.",
                    "ns2.isc-sns.com.",
                    "ns3.isc-sns.info."
                ],
                "rrname": "farsightsecurity.com",
                "rrtype": "NS",
                "time_first": "2013-06-30T16:21:41Z",
                "time_last": "2013-07-18T16:22:47Z"
            },
            {
                "bailiwick": "com",
                "count": 157,
                "from_zone_file": true,
                "rdata": [
                    "ns.sjc1.vix.com.",
                    "ns.sql1.vix.com."
                ],
                "rrname": "farsightsecurity.com",
                "rrtype": "NS",
                "time_first": "2013-01-24T17:18:05Z",
                "time_last": "2013-06-29T16:19:01Z"
            },
            {
                "bailiwick": "com",
                "count": 1890,
                "from_zone_file": true,
                "rdata": [
                    "ns5.dnsmadeeasy.com.",
                    "ns6.dnsmadeeasy.com.",
                    "ns7.dnsmadeeasy.com."
                ],
                "rrname": "farsightsecurity.com",
                "rrtype": "NS",
                "time_first": "2013-07-19T16:22:00Z",
                "time_last": "2020-07-24T16:02:05Z"
            },
            {
                "bailiwick": "farsightsecurity.com",
                "count": 6350,
                "from_zone_file": false,
                "rdata": [
                    "66.160.140.81"
                ],
                "rrname": "farsightsecurity.com",
                "rrtype": "A",
                "time_first": "2013-09-25T15:37:03Z",
                "time_last": "2015-04-01T06:17:25Z"
            },
            {
                "bailiwick": "farsightsecurity.com",
                "count": 36770,
                "from_zone_file": false,
                "rdata": [
                    "104.244.13.104"
                ],
                "rrname": "farsightsecurity.com",
                "rrtype": "A",
                "time_first": "2015-04-01T14:17:52Z",
                "time_last": "2018-09-27T00:29:43Z"
            }
        ]
}

Workflows

• Example Pre-Process Script:

inputs.owner_name = artifact.value

• Example Post-Process Script:

dnsdb_records = results['dnsdb_records']

incident.addNote(helper.createRichText("{0} DNSDB RRSet Records Found".format(str(len(dnsdb_records)))))

Function - DNSDB Rate Limit

This function queries the “rate limit” endpoint to obtain information about the api key’s service limits and quotas.

• See [Farsight DNSDB API Version 2 Documentation]({{ site.baseurl }}/dnsdb/dnsdb-apiv2/#service-limits-and-quotas) section on "service limits and quotas".

Function - DNSDB Pivot

This function allows you to execute a series of queries against DNSDB. Each subsequent search “pivots” on the keys returned by the previous. You may pivot using the rrname, rdata, or raw_rdata field. This function takes a single argument, “pivot”, which is a json-encoded array of dictionaries. Each dictionary contains parameters for a DNSDB search, including a “function” field which may be any of “rdata”, “rrset”, or “flex”. Arguments for each search are the same as their corresponding Resilient function.

Required arguments:

• rrset: owner_name
• rdata: type, value
• flex: key, method, value

Optional arguments:

• limit, offset, time_first_before, time_first_after, time_last_before, time_last_after

The second through last dictionaries must each contain a “pivot” key which may be “rrname”, “rdata”, or “raw_rdata”, and indicates which field from the preceding query should be used as the key for the next.

• Example:

inputs.pivot = """
[
\{\{"function": "rrset", "owner_name": "{0}", "rrtype": "{1}", "limit": {2},
    "time_first_before": "{3}", "time_last_after": "{4}"}},
\{\{"function": "rdata", "type": "ip", "pivot": "rdata"}}
]
""".format(artifact.value, "A", 10, time_first_before, time_last_after)

Additional Information

• Farsight DNSDB API Version 2 Documentation
• Farsight DNSDB Flex Reference Guide

Rules

Rule Name	Object	Workflow Triggered
DNSDB RRSet Lookup	artifact	rrset_workflow
DNSDB RData Lookup	artifact	rdata_workflow
DNSDB Flex Lookup	artifact	flex_workflow
DNSDB Co-Located Host Lookup	artifact	dnsdb_colocated_host_workflow
DNSDB Historical Address Lookup	artifact	dnsdb_historical_address_workflow
DNSDB Historical Hosts Lookup	artifact	dnsdb_historical_host_workflow