Daily Threat Intelligence Feeds User Guide¶
Introduction¶
Access to DomainTools Daily Threat Intelligence Feeds is provided through the Download API. The below table describes and provides a link to more information about each feed.
DomainTools offers the same feeds in real-time format; consult our Real-Time Threat Intelligence Feeds.
Our Threat Intelligence Feeds often include risk scoring. Consult our Domain Risk Score User Guide for more information.
Consult the Feed Paths, Response Structures, and Inclusion Criteria below for additional details.
| GET Endpoint | Feed | Description |
|---|---|---|
5_min_domain_whois, 5_min_domain_whois_parsed |
5-Min Domain WHOIS | The most recently registered or changed domains, processed on a 5-minute basis. Available as raw and parsed records. |
5_min_ip_whois 5_min_ip_whois_parsed |
5-Min IP WHOIS | The most recently updated IPv4 WHOIS records, processed on a 5-minute basis. Available as raw and parsed records. |
daily_domain_discovery |
Daily Domain Discovery | Daily feed of newly registered and newly observed domains. |
daily_domain_hotlist |
Daily Domain Hotlist | Daily feed of high-risk domains that are observed to be active within a 24 hour time window. Requires: proximity score of 70+; Threat Profile score of 90+; pDNS activity within 24 hours. |
daily_domain_risk |
Daily Domain Risk Feed | Daily feed of high risk domains, regardless of observed traffic. Requires: Combined Risk score of 70+. |
daily_ip_hotlist |
Daily IP Hotlist | Daily feed of high-risk IP addresses hosting hostile domains that are observed to be active within a 24 hour time window, with risk scores and other enrichment data. Requires: More than 50% of domains on the IP have proximity score of 70+ or Threat Profile score of 90+; pDNS activity on malicious domains within 24 hours. |
daily_ip_risk |
Daily Hosting IP Risk | Daily feed of all IP addresses known to be hosting domains, with risk scores and other enrichment data. Requires: IP is actively hosting one or more domains (regardless of risk level). |
Requirements¶
Accessing the threat intelligence feeds included in this document requires the following:
- An Enterprise Account with DomainTools, which is accessible at https://account.domaintools.com/my-account/.
- Authentication credentials for HMAC (preferred, more secure) or an API username and key. HMAC code examples are provided in the Appendix.
- A way to interact with a REST API delivered through AWS CloudFront.
Please contact enterprisesupport@domaintools.com for assistance.
Authentication¶
Hash-Based Message Authentication Code (HMAC)¶
HMAC, or Hash-based Message Authentication Code, is our preferred authentication scheme. HMAC is specified in RFC 2104, and is a more secure method than authenticating with an API username and key. If you need to obtain your secret API key or are unsure about using HMAC, please contact enterprisesupport@domaintools.com.
HMAC uses your secret API key to create a hash out of your authentication request string. The request string contains your API username, the current date-time in ISO 8601 with a timezone offset, and the request URI. The authenticating API server uses the same secret API key to hash the request string and authenticates the request if the two hashes match.
HMAC Configuration Requirements¶
- DomainTools supports
MD5,SHA1, andSHA256for the hashing algorithm. - The URI is the part of the URL after api.domaintools.com of your request, e.g.
/v1/example.com/whois - The timestamp value for the authenticating user's hash must be identical to the timestamp parameter passed to the authenticating API server.
HMAC Parameters¶
| Required Parameter | Value |
|---|---|
api_username |
Your API username |
timestamp |
Current timestamp, in ISO 8601 format, e.g.: 2024-07-01T14:37:59-0800 2024-07-01T22:37:59Z 2024-07-02T10:37:59+1200 |
signature |
HMAC signature of your request, using the MD5, SHA1, or SHA256 hashing algorithm |
Open Key Authentication¶
This is the easiest authentication scheme to implement, but also the least secure. Refer to the Hash-Based Message Authentication Code (HMAC) section of this document to configure the more secure authentication method, or contact enterprisesupport@domaintools.com for help.
Open Key Parameters¶
| Required Parameter | Value |
|---|---|
api_username |
Your API username |
api_key |
Your API key |
Endpoints and Parameters¶
Base URL¶
Consult the Feed Paths, Response Structures, and Inclusion Criteria below for additional details.
Feed Path Parameters¶
| Path Parameter | Feed Name | Feed Description |
|---|---|---|
daily_domain_hotlist |
Daily Domain Hotlist | Daily feed of high-risk domains that are observed to be active within a 24 hour time window. Requires: proximity score of 70+; Threat Profile score of 90+; pDNS activity within 24 hours. |
daily_domain_risk |
Daily Domain Risk Feed | Daily feed of high-risk domains, regardless of observed traffic. Requires: Combined Risk score of 70+. |
daily_domain_discovery |
Daily Domain Discovery | Daily feed of newly registered and newly observed domains. |
daily_ip_hotlist |
Daily IP Hotlist | Daily feed of high-risk IP addresses hosting hostile domains that are observed to be active within a 24 hour time window, with risk scores and other enrichment data. Requires: More than 50% of domains on the IP have proximity score of 70+ or Threat Profile score of 90+; pDNS activity on malicious domains within 24 hours. |
daily_ip_risk |
Daily Hosting IP Risk | Daily feed of all IP addresses known to be hosting domains, with risk scores and other enrichment data. Requires: IP is actively hosting one or more domains (regardless of risk level). |
5_min_domain_whois 5_min_domain_whois_parsd |
5-Min Domain WHOIS | The most recently registered or changed domains, processed on a 5-minute basis. |
5_min_ip_whois 5_min_ip_whois_parsed |
5-Min IP WHOIS | The most recently updated IPv4 WHOIS records, processed on a 5-minute basis. |
Common Query Parameters¶
| Parameter | Description | Type | Required? |
|---|---|---|---|
api_username |
API username provided by DomainTools | string | Yes |
api_key |
API key provided by DomainTools, dashes included | string | Yes (for key auth) |
timestamp |
Current timestamp for HMAC authentication, in ISO 8601 format. (e.g. 2025-01-10T15:44:39.118Z) | string | Yes (for HMAC auth) |
signature |
HMAC hash of your request, using the MD5, SHA1, or SHA256 hashing algorithm |
string | Yes (for HMAC auth) |
limit |
Limit the list of signed files. Ordering of files is always descending, so the latest files are first. Large reports like 5_min_parsed_whois can have over 2k files (5 min files for 7 days). By using the limit parameter, clients can limit the lookup to an arbitrary number (integer) of files. |
integer | No |
app_partner |
Your product name. Useful to help debugging | string | No |
app_name |
Name of your appliance, playbook, module, etc., or in combination. Useful to help with debugging | string | No |
app_version |
Your version number. Useful to help with debugging | string | No |
Common Response Parameters¶
| Parameter | Description | Example | Type |
|---|---|---|---|
download_name |
The feed path of the download. | 5_min_domain_whois 5_min_domain_whois_parsed 5_min_ip_whois 5_min_ip_whois_parsed daily_domain_discovery daily_domain_hotlist daily_domain_risk daily_ip_hotlist daily_ip_risk |
string |
name |
A feed directory containing a feed file | domain_discovery/NEW_DOMAINS_20240704.csv.gz |
string |
last_modified |
Last modified date of the file in ISO 8601 format. | 2024-07-05T17:27:18+00:00 |
string |
etag |
Entity tag (a hash of the download file). | \"595if37f25666447bd45b81fef3aaee5\" |
string |
size |
Size in kilobytes (KB) of the download file | string | |
url |
Signed AWS CloudFront download URL | string |
Query Example¶
Response Example¶
In response, api.domaintools.com will return a download payload with either a single or an array of signed CloudFront URLs. URLS are valid for 12 hours (and can be re-generated). For example:
{
"response": {
"download_name": "daily_ip_risk",
"files": [
{
"name": "daily_ip_risk/ip_fulllist.gz",
"last_modified": "2024-11-12T21:32:43+00:00",
"etag": "\"43db708555fec88f423c4751ffa2d2b6\"",
"size": "197407887",
"url": "https://d2mzrdiuqyylox.cloudfront.net/daily_ip_risk/ip_fulllist.gz?Expires=1731562356&Signature=SCYQZkHzZvBZ30TiA5XO8~Tulv0B4Ep3gIRKPyYN97RbelUIiSUeC2g6D5lAzmdHzBBjR5PThZ2zdkn7TehgO7L8MclMgwZYKqydUsCZxGGsafxqXEtR6bLDq72qBRgtwln5LbrC0N1WjnVkVaPdtqA3pmeAk3bRJTfqjDoEUywIOAIotMBN7Vbm2ZcFvY4C1Xsyf53Z~cgYjeTR9g5OALsjNPdTYqQIWC2oWI2CHQQMj13wubvS4ZgorEFdljreB6~TOvI5M6XuAJoePpjeirrD76xh7GpektACdUKMpgCu-2VnJC~UScYq6sCMHt-ZAvWnB5CuCkGolSuiM9zZZQ__&Key-Pair-Id=KJPH4C4RWZJ32"
}
]
}
}
Feed Paths, Response Structures, and Inclusion Criteria¶
Daily Domain Hotlist¶
The Daily Domain Hotlist is also available in RPZ format, and documentation for this method is pending. In the meantime please contact enterprisesupport@domaintools.com for information on this method.
| Description | Inclusion Threshold | Format | Size |
|---|---|---|---|
| Daily feed of high-risk domains that are observed to be active within a 24 hour time window. | Proximity score of 70+; Threat Profile score of 90+; pDNS activity within 24 hours. | Tab separated text file (gzipped); one domain per line with component risk scores. | \~900,000 domains, \~3.5MB compressed. |
Daily Domain Hotlist: Endpoint¶
Consult Common Query Parameters for more details about how to construct your request.
Daily Domain Hotlist: Response Fields¶
| Response Field | Description |
|---|---|
| Domain Name | Pinpoint spoof or otherwise suspicious domain names |
| Phishing | Machine learning classifier prediction for phishing |
| Malware | Machine learning classifier prediction for malware |
| Spam | Machine learning classifier prediction for spam |
| Proximity | Indicates shared registration or infrastructure with known-bad domains |
| Overall | Equals highest of the component scores |
Daily Domain Hotlist: Files¶
domain_hotlist.gz
Daily Domain Risk Feed¶
| Description | Inclusion Threshold | Format | Size |
|---|---|---|---|
| Daily feed of high risk domains, regardless of observed traffic. | Combined Risk score of 70+. | Tab separated text files (gzipped); one domain per line with component risk scores. | \~30 million domains, \~400MB compressed. |
Daily Domain Risk Feed: Endpoint¶
Consult Common Query Parameters for more details about how to construct your request.
Daily Domain Risk Feed: Response Fields¶
| Response Field | Description |
|---|---|
| Domain Name | Pinpoint spoof or otherwise suspicious domain names |
| Phishing | Machine learning classifier prediction for phishing |
| Malware | Machine learning classifier prediction for malware |
| Spam | Machine learning classifier prediction for spam |
| Proximity | Indicates shared registration or infrastructure with known-bad domains |
| Overall | Equals highest of the component scores, the “combined” score |
Daily Domain Risk Feed: Files¶
domain_risk_feed/threat_profile_proximity.gz
Daily Domain Discovery¶
| Description | Inclusion Threshold | Format | Size |
|---|---|---|---|
| Daily feed of newly registered and newly observed domains. | Newly observed or registered. | gzip-compressed csv file; one domain name per line. | \~375,000 domains, \~2.5MB compressed. |
Daily Domain Discovery: Endpoint¶
Consult Common Query Parameters for more details about how to construct your request.
Daily Domain Discovery: Response Fields¶
One domain name per line.
Daily Domain Discovery: Files¶
NEW_DOMAINS_YYYYMMDD
Daily IP Hotlist¶
| Description | Inclusion Threshold | Format | Size |
|---|---|---|---|
| Daily feed of high-risk IP addresses hosting hostile domains that are observed to be active within a 24 hour time window, with risk scores and other enrichment data. | More than 50% of domains on the IP have proximity score of 70+ or Threat Profile score of 90+; pDNS activity on malicious domains within 24 hours. | zip-compressed tab separated text file; one IPv4 per line with percentages of phishing, malware, and spam metrics for domains hosted in the IP. | 40-50,000 IP addresses, ~1MB compressed. |
Daily IP Hotlist: Endpoint¶
Consult Common Query Parameters for more details about how to construct your request.
Daily IP Hotlist: Response Fields¶
| Field | Description |
|---|---|
| Threat Type | Understand the risk category of domains on the IP |
| ISP and Geolocation | Confirm geographical attributes and ownership |
| Domain Stats | Measure the IP’s reach and scale |
| Confirmed Threats | Obtain more granular details on “convicted” domains on the IP |
| Predicted Threats | Threat predictions for domains not yet found on industry blocklists |
| pDNS and Zerolisted Metrics | Avoid false positives with allow-listed domains; scope traffic activity as recorded by worldwide passive DNS sensors |
Daily IP Hotlist: Files¶
ip_hotlist.gz
Daily Hosting IP Risk Feed¶
| Description | Inclusion Threshold | Format | Size |
|---|---|---|---|
| Daily feed of all IP addresses known to be hosting domains, with risk scores and other enrichment data. | IP is actively hosting one or more domains (regardless of risk level). | gzip-compressed tab separated text file; one IPv4 per line with additional fields (consult full entry). | 15-20 million IP addresses, \~200MB compressed. |
Daily Hosting IP Risk Feed: Endpoint¶
Consult Common Query Parameters for more details about how to construct your request.
Daily Hosting IP Risk Feed: Response Fields¶
| Field Name | Field Description |
|---|---|
| 3rdparty_threats | Number of domains on IP that are confirmed with any threat on 3rd party intel feed |
| allthreats_combined_percent | Percentage of domains which are confirmed or predicted malicious |
| asn | The IP's ASN (i.e. routing provider) |
| bad_pdns_resolutions | Number of confirmed bad domains seen on the IP in the last 24 hours |
| city | City |
| combined_malware_percent | Percentage of domains confirmed or predicted as malware |
| combined_spam_percent | Percentage of domains confirmed or predicted as spam |
| combined__phishing_percent | Percentage of domains confirmed or predicted as phishing |
| country | Country |
| ip | IP with www/apex domains pointing to it |
| latitude | Coordinates |
| longitude | Coordinates |
| organization | ASN provider name or organization |
| pdns_resolutions | Number of domains seen on the IP in the last 24 hours |
| region | Region |
| total_domains | Total number of domains see on the IP in the last 30 days |
| allthreats_combined_count | Number of confirmed or predicted domains on 3rd party intel feed or Threat Profile |
| malicious_phishing | Number of malicious phishing domains on 3rd party intel feeds |
| malicious_malware | Number of malicious malware domains on 3rd party intel feeds |
| malicious_spam | Number of malicious spam domains on 3rd party intel feeds |
| compromised_phishing | Number of compromised phishing domains on 3rd party intel feeds |
| compromised_malware | Number of compromised malware domains on 3rd party intel feeds |
| compromised_spam | Number of compromised spam domains on 3rd party intel feeds |
| predicted_phishing | Number of domains (with no confirmed threat) Threat Profile predicts as phishing |
| predicted_malware | Number of domains (with no confirmed threat) Threat Profile predicts as malware |
| predicted_spam | Number of domains (with no confirmed threat) Threat Profile predicts as spam |
| percent_phishing | Percentage of domains that are confirmed phishing |
| percent_malware | Percentage of domains that are confirmed malware |
| percent_spam | Percentage of domains that are confirmed spam |
| zerolist_domains | Number of zerolisted domains seen on this IP |
| zerolist_ip | Indicates if this IP is zerolisted by DomainTools |
Daily Hosting IP Risk Feed: Files¶
ip_fulllist.gz
Daily 5-Minute Domain WHOIS Feed¶
| Description | Inclusion Threshold | Format | Size |
|---|---|---|---|
| The most recently updated domain WHOIS records, processed on a 5-minute basis. | All domain names processed since the previous update. | gzip-compressed tab separated or JSON text file. | Up to 10MB per day |
Daily 5-Minute Domain WHOIS Feed: Endpoint¶
api.domaintools.com/v1/download/5_min_domain_whois
api.domaintools.com/v1/download/5_min_domain_whois_parsed
Consult Common Query Parameters for more details about how to construct your request.
Daily 5-Minute Domain WHOIS Feed: Response Fields¶
| Domain WHOIS Response Fields |
|---|
| Domain name |
| Parse success (y/n) |
| Server (WHOIS) |
| Lookup Date |
| Lookup Time |
| Create Date |
| Updated Date |
| Expires Date |
| Registrar Name |
| Registrar Abuse Contact: Phone |
| Registrar Abuse Contact: Email |
| Registrar IANA ID |
| Registrar URL |
| Registrar WHOIS Server |
| Admin Name |
| Admin Org |
| Admin Street |
| Admin City |
| Admin State/Province |
| Admin Postal Code |
| Admin Country |
| Admin Phone |
| Admin Fax |
| Admin Email |
| Billing Name |
| Billing Org |
| Billing Street |
| Billing City |
| Billing State/Province |
| Billing Postal Code |
| Billing Country |
| Billing Phone |
| Billing Fax |
| Billing Email |
| Registrant Name |
| Registrant Org |
| Registrant Street |
| Registrant City |
| Registrant State/Province |
| Registrant Postal Code |
| Registrant Country |
| Registrant Phone |
| Registrant Fax |
| Registrant Email |
| Technical Name |
| Technical Org |
| Technical Street |
| Technical City |
| Technical State/Province |
| Technical Postal Code |
| Technical Country |
| Technical Phone |
| Technical Fax |
| Technical Email |
| Name Server |
| Registrar Status |
| Raw WHOIS Data Blob |
Daily 5-Minute Domain WHOIS Feed: Files¶
YYYYMMDDTTTT.json.gz \= the parsed feed file
YYYYMMDDTTTT.gz \= the raw/unparsed feed file
Daily 5-Minute IP WHOIS Feed¶
| Description | Inclusion Threshold | Format | Size |
|---|---|---|---|
| The most recently updated WHOIS records are processed on a 5-minute basis. | All IP addresses processed since the previous update. | gzip-compressed tab separated or JSON text file. | Up to 10MB per day |
Daily 5-Minute IP WHOIS Feed: Endpoint¶
api.domaintools.com/v1/download/5_min_ip_whois
api.domaintools.com/v1/download/5_min_ip_whois_parsed
Consult Common Query Parameters for more details about how to construct your request.
Daily 5-Minute IP WHOIS Feed: Response Fields¶
| IP WHOIS Response Fields |
|---|
| RIR queried |
| Net Range |
| CIDR |
| Net Name |
| Net Handle |
| Parent |
| Net Type |
| Origin AS |
| Organization |
| RegDate |
| Updated |
| Org Name |
| Org ID |
| City |
| State |
| Postal Code |
| Country |
| RegDate |
| Updated |
| Ref |
| Referral Server |
| OrgAbuseHandle |
| OrgAbuseName |
| OrgAbusePhone |
| OrgAbuseEmail |
| OrgAbuseRef |
| OrgTechHandle |
| OrgTechName |
| OrgTechPhone |
| OrgTechEmail |
| OrgTechRef |
| OrgNOCHandle |
| OrgNOCName |
| OrgNOCPhone |
| OrgNOCEmail |
| OrgNOCRef |
| Comments |
| Raw IP WHOIS data blob |
Daily 5-Minute IP WHOIS Feed: Files¶
YYYYMMDDTTTT.json.gz \= the parsed feed file
YYYYMMDDTTTT.gz \= the raw/unparsed feed file