Splunk Enterprise app configuration tables and troubleshooting

This reference provides detailed configuration tables, troubleshooting guidance, and historical release notes for the DomainTools App for Splunk.

Configuration tables

Main configuration files

These configuration files are relevant for using the app and DomainTools datasets.

Note

The configuration files are relevant for this version only. The configuration files, stanzas, and fields are different in other versions.

app.conf

Add details for the Splunk App.

Stanza	Fields
package	id
install	is_configured
ui	is_visible, label
launcher	author, description, version
triggers	reload.domaintools

commands.conf

Helper commands for the app. The most commonly used commands outside the app are described in greater detail in Search and in the in-app documentation. Every stanza sets python.version = python3 and python.required = 3.13. The chunked parameter indicates the search command supports Splunk’s “chunked” custom protocol; the filename parameter is the Python .py file that implements the command.

Stanza	Fields
dtaccountinfo	chunked, filename
dtimportirisdetectmonitors	chunked, filename
dtimportirisdetectresults	chunked, filename
dtirisdetectescalate	chunked, filename
dtirisdetectchangestate	chunked, filename
dtsyncirisdetectwatchlist	chunked, filename
dtirisinvestigate	chunked, filename
dtirisenrich	chunked, filename
dtwhoishistory	chunked, filename
dtdnsdbenrich	chunked, filename
dtformatenrich	chunked, filename
dtformatinvestigate	chunked, filename
dtdomainextract	chunked, filename
dtdomainrefang	chunked, filename
dtidnadecode	chunked, filename
dtfeednod	chunked, filename
dtfeednad	chunked, filename
dtfeednoh	chunked, filename
dtfeeddomainrdap	chunked, filename
dtfeeddomaindiscovery	chunked, filename
dtparseddomainrdap	chunked, filename
dtfeeddomainrisk	chunked, filename
dtfeeddomainhotlist	chunked, filename
dtfeediphotlist	chunked, filename
dtfeediprisk	chunked, filename
dtexpirecache	chunked, filename
dtdnsdb	filename, retainsevents, supports_multivalues, streaming, overrides_timeorder, passauth
dtdnsdbflex	filename, retainsevents, supports_multivalues, streaming, overrides_timeorder, passauth
validateip	filename, retainsevents, supports_multivalues, streaming, overrides_timeorder
dtdnsdblimit	filename, retainsevents, supports_multivalues, streaming, overrides_timeorder, passauth
flushcache	filename, retainsevents, supports_multivalues, streaming, overrides_timeorder, passauth

The commands.conf file also carries a commented template showing how to switch dtdomainextract to the Splunk SDK SCP1 protocol (type = python, streaming = true, local = true, passauth = true, chunked = false) if the current SDK hits throughput issues.

searchbnf.conf

The syntax (shorter name), description, and whether the usage is public.

Stanza	Fields
dtaccountinfo-command	syntax, shortdesc, usage, comment1, example1
dtimportirisdetectmonitors-command	syntax, shortdesc, usage, comment1, example1, related
dtimportirisdetectresults-command	syntax, shortdesc, usage, comment1, example1, related
dtirisinvestigate-command	syntax, shortdesc, description, usage, comment1, example1, comment2, example2, comment3, example3, related
dtirisenrich-command	syntax, shortdesc, description, usage, comment1, example1, comment2, example2, related
dtwhoishistory-command	syntax, shortdesc, description, usage, comment1, example1, comment2, example2
dtformatinvestigate-command	syntax, shortdesc, description, usage, comment1, example1, related
dtformatenrich-command	syntax, shortdesc, usage
dtdomainextract-command	syntax, shortdesc, description, comment1, example1, comment2, example2, usage
dtsyncirisdetectwatchlist-command	syntax, shortdesc, usage, comment1, example1
dtexpirecache-command	syntax, shortdesc, usage, comment1, example1
dtdnsdb-command	syntax, shortdesc, description, example1, example2, example3, example4, usage
dtdnsdbflex-command	syntax, shortdesc, description, example1, example2, example3, usage
dtdnsdblimit-command	syntax, description, shortdesc, example1, usage
dtdnsdbenrich-command	syntax, description, shortdesc, example1, comment1, example2, comment2, example3, comment3, usage
dtidnadecode-command	syntax, shortdesc, comment1, example1, usage
dtfeednod-command	syntax, shortdesc, example1, usage
dtfeednad-command	syntax, shortdesc, example1, usage
dtfeednoh-command	syntax, shortdesc, example1, usage
dtfeeddomainrdap-command	syntax, shortdesc, example1, usage
dtfeeddomaindiscovery-command	syntax, shortdesc, example1, usage
dtparseddomainrdap-command	syntax, shortdesc, example1, usage
dtfeeddomainrisk-command	syntax, shortdesc, example1, usage
dtfeeddomainhotlist-command	syntax, shortdesc, example1, usage
dtfeediphotlist-command	syntax, shortdesc, example1, usage
dtfeediprisk-command	syntax, shortdesc, example1, usage

server.conf

Stanza	Fields	Description
shclustering	conf_replication_include.domaintools	Default value is set to true.

transforms.conf

KV store fields. See the KV store table for the array of fields_list for each stanza.

Stanza	Fields
dt_iris_enrich_queue	external_type, collection, fields_list, case_sensitive_match
dt_iris_enrich_data	external_type, collection, fields_list, case_sensitive_match
dt_stats	external_type, collection, fields_list, case_sensitive_match
dt_allowlist	external_type, collection, fields_list
dt_monitoring_list	external_type, collection, fields_list
dt_tags_list	external_type, collection, fields_list
dt_iris_detect_monitors	external_type, collection, fields_list
dt_iris_detect_results	external_type, collection, fields_list
dt_iris_investigate	external_type, collection, fields_list
dt_public_suffix_list	filename, match_type, max_matches

domaintools.conf

All stanzas use the domaintools stanza tag.

Field	Description
proxy_enabled	Use a proxy when connecting to the DomainTools API. To enable, set to 1.
proxy_server	The proxy server address to use.
proxy_port	The proxy server port to use.
proxy_authentication	Whether the proxy requires authentication. To enable, set to 1.
custom_certificate_enabled	Use a custom SSL certificate for the SSL connection. To enable, set to 1.
custom_certificate_path	The path to the custom SSL certificate.
guided_pivot_threshold	The Guided Pivot Threshold on the Domain Profile page. Set a lower value to narrow investigations. 500 is the default and recommended value.
bulk_enrichment_batch_size	Number of domains batched in an API call. Set the value from 1 to 100.
optimize_enrichment_searches	Enables quicker correlation of cached data of known domains from the Enrichment table. Requires additional disk space. Disabling reduces disk space consumption but slows down searches. Set 1 to enable.
populate_scores	Whether to use the Risk Score over lower tiered scores. Turned off (set to 0) by default.
logging_on	Toggles whether to write logs to file.

Other configuration files

Conf File	Stanza	Fields	Description
macros.conf	See Key macros for enrichment
savedsearches.conf	See Configure Saved Searches
collections.conf	See KV store names and fields
distsearch.conf	replicationWhitelist	domainextract	Path to the domain_extract.py custom search command to copy to indexers.
		idnadecode	Path to the idna_decode.py custom search command to copy to indexers.
		lib	Path to Python libs to copy to indexers.
		allConf	Path to the app’s default configuration to copy to indexers.
workflow_actions.conf		dt_iris_lookup	Look up the domain in Iris Investigate (link action).
		dt_domain_profile	Open the domain in the app’s Domain Profile page (link action).
		dt_dnsdb	Look up passive DNS in the app’s DNSDB page (link action).
		dt_related_events	Search related events with | dt_basesearch over the last 24 hours (search action).

KV store names and fields

dt_iris_enrich_queue

_key, domain, queued, observed

dt_iris_enrich_data

Fields (120+)

_key, _raw, dt_queued, dt_retrieved, dt_observed, dt_event_seen, dt_unknown, en_domain_name, en_is_active, en_adsense_code, en_google_analytics_code, en_popularity_rank, en_domain_create_date, en_domain_updated_timestamp, en_domain_expiration_date, en_first_seen, en_server_type, en_website_title, en_tld, en_website_response_code, en_redirect_url, en_registrant_name, en_registrant_org, en_registrar, en_spf_info, en_ga4, en_gtm_codes, en_fb_codes, en_hotjar_codes, en_baidu_codes, en_yandex_codes, en_matomo_codes, en_statcounter_project_codes, en_statcounter_security_codes, en_additional_codes_raw, en_additional_whois_email, en_additional_soa_email, en_additional_ssl_raw, en_ssl_info_1_hash, en_ssl_info_1_organization, en_ssl_email, en_ssl_info_1_subject, en_ssl_info_issuer_common_name, en_ssl_info_common_name, en_ssl_info_not_after, en_ssl_info_not_before, en_ssl_info_duration, en_ssl_info_alt_names, en_risk_score, en_proximity_score, en_threat_profile_type, en_threat_profile_malware, en_threat_profile_phishing, en_threat_profile_spam, en_threat_profile_evidence, en_additional_name_servers_raw, en_name_server_1_domain, en_name_server_1_host, en_name_server_1_ip, en_name_server_2_domain, en_name_server_2_host, en_name_server_2_ip, en_additional_mx_raw, en_mx_1_domain, en_mx_1_host, en_mx_1_priority, en_mx_1_ip, en_additional_ips_raw, en_ip_1_address, en_ip_1_country_code, en_ip_1_isp, en_ip_1_asn, en_ip_2_address, en_ip_2_country_code, en_ip_2_isp, en_ip_2_asn, en_admin_contact_city, en_admin_contact_country, en_admin_contact_fax, en_admin_contact_name, en_admin_contact_org, en_admin_contact_phone, en_admin_contact_postal, en_admin_contact_state, en_admin_contact_street, en_admin_contact_email, en_billing_contact_city, en_billing_contact_country, en_billing_contact_fax, en_billing_contact_name, en_billing_contact_org, en_billing_contact_phone, en_billing_contact_postal, en_billing_contact_state, en_billing_contact_street, en_billing_contact_email, en_technical_contact_city, en_technical_contact_country, en_technical_contact_fax, en_technical_contact_name, en_technical_contact_org, en_technical_contact_phone, en_technical_contact_postal, en_technical_contact_state, en_technical_contact_street, en_technical_contact_email, en_registrant_contact_city, en_registrant_contact_country, en_registrant_contact_fax, en_registrant_contact_name, en_registrant_contact_org, en_registrant_contact_phone, en_registrant_contact_postal, en_registrant_contact_state, en_registrant_contact_street, en_registrant_contact_email, en_tag, en_tag_raw

dt_stats

_key, dt_last_enriched_datetime, dt_num_of_times_enriched, dt_num_of_AdhocLookups, dt_fooyn_timestamp, dt_looyn_timestamp, en_attribute_name, en_attribute_type, en_risk_score

dt_allowlist

_key, en_attribute_type, _dt_updated, _dt_updated_by, _dt_created, _dt_created_by

dt_monitoring_list

_key, en_attribute_type, _dt_updated, _dt_updated_by, _dt_created, _dt_created_by, _dt_source

dt_tags_list

_key, en_attribute_type, _dt_updated, _dt_updated_by, _dt_created, _dt_created_by

dt_iris_detect_monitors

_key, monitor_id, term, state, match_substring_variations, nameserver_exclusions, text_exclusions, created_date, updated_date, status, created_by, discover_new_domains, dt_updated

dt_iris_detect_results

_key, dt_domain, dt_state, dt_status, dt_discovered_date, dt_changed_date, dt_escalations, dt_risk_score, dt_risk_status, dt_mx_exists, dt_tld, dt_domain_id, dt_monitor_ids, dt_create_date, dt_ip_address_1, dt_ip_address_2, dt_ip_raw, dt_nameServer_1, dt_nameServer_2, dt_nameServer_raw, dt_mailServer_1, dt_mailServer_2, dt_mailServer_raw, dt_registrar, dt_registrant_contact_email, dt_proximity_score, dt_threat_profile_malware, dt_threat_profile_phishing, dt_threat_profile_spam, dt_threat_profile_evidence, dt_monitor_flag, dt_imported

dt_iris_investigate

_key, dt_pivot_type, dt_pivot_value, dt_investigate_raw, _dt_created

dt_rrset_kvstore

No fields defined.

dt_rdata_kvstore

No fields defined.

Key macros for enrichment

Macro Field Name	Default Value	Description
dt_basesearch		The defined value is the base search. The system pulls data directly from the datamodel. We use this search to search for and queue up domains for the app and certain features such as the dashboards.
enable_cache	1 (enabled)	Enrichment setting to determine caching of enriched data. DomainTools will always enrich every domain in the queue. When turned off (set to 0), the system makes an API call for every domain.
dt_cache_retention_period	30 (in days)	Enrichment setting. Set the value to how many days back before removing older data from the enrichment kvstore. There is also a saved search that will remove records that are over 30 days old.
dt_proximity_score_threshold	65	Enrichment setting. Set the threshold throughout the app when filtering based on the Proximity score.
dt_threat_profile_score_threshold	85	Enrichment setting. Set the threshold throughout the app when filtering based on the Threat Profile score.
dt_high_risk_threshold	90	Enrichment setting. Set the threshold throughout the app.
dt_medium_risk_threshold	70	Enrichment setting. Set the threshold throughout the app.
dt_refresh_interval	15 (in minutes)	The refresh interval.
dtdomainextract2	See note below table	Alternative to dtdomainextract that does regular expression-based matching for TLDs. It is higher performance for high-throughput environments, with a small accuracy trade-off. Notably, some multi-level tlds (for example, edu.np) can be mis-identified as a domain.
dt_risk_score_threshold	75	Enrichment setting. Set the threshold throughout the app when filtering based on the Risk Score.
dt_risk_score_increase_alert_threshold	50	Alert setting. The Risk Score increase (0–100) that triggers the Risk Score Increase alert.
dt_young_domain_age	7 (in days)	Enrichment setting. The number of days the app considers a domain to be young.
dt_include_allowlisted_domains	0 (false)	Allowlist setting. Set to 1 (enabled) to exclude showing domains in the allowlist in our dashboards.
dt_include_monitoring_list_domains	0 (false)	Setting to include monitoring list domains.
dt_enrich_to_stats_lookup		A partial search that the saved searches use to update the enriched data KV Store.
dt_include_allowlisted_domains_in_enrichment	1 (enabled)	Enrichment setting. Whether to include allowlisted domains during enrichment.
dt_include_allowlisted_domains_in_notable_events	0 (false)	Enrichment alert setting for notable events.
dt_only_monitored_domains_in_notable_events	1 (enabled)	Enrichment alert setting for notable events.
dt_use_risk_threshold_in_notable_events	0 (false)	Enrichment alert setting for notable events.
dt_use_threatprofile_threshold_in_notable_events	0 (false)	Enrichment alert setting for notable events.
dt_ignore_iris_detect_in_notable_events	0 (false)	Enrichment alert setting for notable events.
dt_monitor_tags_in_notable_events	0 (false)	Enrichment alert setting for notable events.
dt_exceed_domain_age_threshold_in_young_rule	1 (enabled)	Young Domains rule setting. Whether the domain age threshold applies in the rule.
dt_only_monitored_domains_in_young_rule	0 (false)	Young Domains rule setting. Restrict the rule to monitored domains.
dt_risk_score_increase_in_rule	1 (enabled)	Risk Score Increase rule setting. Enable the Risk Score increase condition in the rule.
dt_use_risk_threshold_in_risk_rule	0 (false)	Risk Score Increase rule setting. Apply the Risk Score threshold in the rule.
dt_use_risk_score_increase_alert_threshold_in_risk_rule	0 (false)	Risk Score Increase rule setting. Apply dt_risk_score_increase_alert_threshold in the rule.
dt_detect_in_rule	1 (enabled)	Iris Detect rule setting. Include Iris Detect results in the rule.
dt_notable_events	See definition below	Search for notable events provided by the DomainTools App for Splunk ES.
dt_rename_base_fields		Renames the base search fields. For example, rename src to Source, dest as Destination, log_source as Log Source and domain as Domain Name.
dt_rename_iris_fields
unknown_domain_retry	1 (enabled)	Retry enrichment of domains that are unknown to DomainTools.
unknown_domain_retry_time	60 (in minutes)	Number of minutes to wait before trying to re-enrich a domain.
toEpoch(1)	if(isnull(round(relative_time(time(), “$reltime$”))), “$reltime$”, round(relative_time(time(), “$reltime$“)))	Changes timestamp to epoch.

dtdomainextract2 default value:

rex field=url "(.*:\/\/)?(?P<temp_domain>[^:#\/?]+)" \
| lookup dt_public_suffix_list wildcard_tld AS temp_domain OUTPUT tld AS tld \
| where match(temp_domain, "(.*[.|@])?([\p{L}\w-]+[.]".tld."$)") \
| eval domain = replace(temp_domain, "(.*[.|@])?([\p{L}\w-]+[.]".tld."$)", "\2")

dt_notable_events default value:

search index=notable | eval count=1 | where search_name IN ("DomainTools - DomainTools Young Domains - Rule", "DomainTools - DomainTools Domain Monitoring - Rule")

Troubleshooting

Splunk Cloud configuration during install

The self-service app installation might run into issues installing components required on indexers. The installation proceeds normally, but attempts to run dtdomainextract return an error. This may be due to Splunk self-service only installing apps on search heads.

Check the status of saved searches

Having one or more required saved searches turned off is a common customer issue that could manifest itself as incomplete app functionality.

To check on the status of saved searches:

1. Select the DT Settings menu within the app.
2. Select Configure Saved Searches to load the list of saved searches used by the DomainTools app

Compare the scheduled time on the resulting set of reports against the table of Saved Search Names and Descriptions to ensure the required core app saved searches, as well as the additional ones if required for Iris Investigate or Alerting in Splunk Enterprise Security are enabled.

Non-production environments

Validating the app in non-production environments: if you use a staging environment or development environment to test new Splunk apps, ensure the same data sources you plan to use in production are also available to the Splunk search heads in the test environment.

System architecture

Architecture overview

High-level topology of both Splunk and DomainTools resources

The Saved Searches configuration file (savedsearches.conf) defines the processes for enrichment and the Queue Builder for the Iris Enrich KV store. In the Queue Builder process, the system queries raw logs in the Splunk Indexes from the Web data model as the DomainTools base search configuration (dt_basesearch) defines.

This process includes checking to see if the domain already exists when comparing to existing Iris Enrich data, as that would indicate if DomainTools already enriched the domain. If not, the system queues the new domain for enrichment. The KV store stores each domain with the enriched data.

Domain Enrichment Process between DomainTools and Splunk Indexes

Prior 5.x release notes

Release notes for version 5.3

New in version 5.3

• Support for Newly Active Domains (NAD) and Newly Observed Domains (NOD) Threat Feeds

Fixed in version 5.3

• Public suffix list in Enrichment Summary
• Guided pivots failing for email addresses
• Newly observed domains race condition

Release notes for version 5.2

New in version 5.2

• Enrichment Explorer visualization and filters
• Enrichment dashboard panels: Top ASN; Top Registrars; Top SSL Expired Certificates; Top Nameservers; Top ISPs; Top IP

Updated in version 5.2

• Additional fields in Enrichment Explorer summary view
• Error handling for Iris Investigate
• Allow multi-value inputs to dtirisenrich command

Fixed in version 5.2

• dtwhoishistory documentation
• DNSDB error message specificity

Release notes for version 5.1

Updated in version 5.1

• Improved API usage reporting
• Improved information and responses around required API products

New in version 5.1

• Support for new Iris fields

Release notes for version 5.0

New in version 5.0

• Send alerts sent to any SOAR platform with the new dt_alerts index
• New configurable Risk Score Increase alert from Iris Detect results.
• Log all domain enrichment values, and compare enrich values over time, using the new dt_enrich_history index.
• Track changes to WHOIS data with the dtwhoishistory custom search script
• Iris Investigate and Enrich API responses now include website_title, first_seen and server_type, as well as the SSL fields ssl.alt_names, ssl.duration, ssl.common_name, ssl.issuer_common_name, ssl.not_after, and ssl.not_before. These are available in the domain profile, enrichment explorer, and in Enrich and Investigate custom search commands.