DNSDB service limits and quotas

The DNSDB API implements quota management to control usage and ensure fair access to resources.

Concurrent Connections

The number of concurrent connections to a DNSDB API server may be limited. This limit is separate from the quota limit described below.

Quota Types

API keys have a primary quota, which limits the number of requests that can be made to the data-fetching endpoints (lookup, /dnsdb/v2/lookup, summarize, and /dnsdb/v2/summarize). There are three types of quotas:

Time-Based Quotas

Time-based quotas are usually applied on a daily basis and reset daily at 00:00 (midnight) in the UTC time zone. Time-based quotas can also be applied for arbitrary time-quantum, but this is less common.

Block-Based Quotas

For block quotas, there are a number of queries that are bought, with all, or a subset, of that number “split” off for a particular API key. Block quotas have:

A specific number of queries split with an expiration time (usually much longer than a day)
The DNSDB API server tracks the number of queries split, number used, and the expiration time
The number of queries bought is not visible to the DNSDB API server

Unlimited Quotas

Unlimited quotas do not limit the number of queries per day.

Burst Rate Quota

There may also be a secondary, burst rate, quota associated with an API key. The burst rate limits how many requests may be made in a short time window.

Example: 5 requests in 360 seconds

The parameters for burst rate are:

burst_size: Maximum number of requests in the window
burst_window: Time window in seconds

Checking Your Quota

You may query the /dnsdb/v2/rate_limit endpoint to obtain a JSON map containing quota information. Querying the /dnsdb/v2/rate_limit endpoint does not count against the quota limit.

Rate Limit Response

The /dnsdb/v2/rate_limit endpoint returns a JSON map named rate containing some or all of the following:

Key	Description
limit	The maximum number of API lookups that may be performed. This is the initial quota.
remaining	For time-based quotas: the remaining number of API lookups that may be performed until the reset time. For block-based quotas: the remaining number of API lookups in the block quota.
reset	For time-based quotas: UNIX epoch timestamp with second granularity indicating the next point in time when the quota limit will be reset. Usually this is at 00:00 (midnight) UTC. For block-based quotas: the value will be “n/a”.
expires	Only present for block-based quota: UNIX epoch timestamp with second granularity indicating when the quota will expire.
results_max	Returns the maximum number of results that can be returned by these lookup methods. This overrides a “limit” query parameter if provided. For example, if “?limit=20000” is appended to the URL path but results_max=1000 then only up to 1000 results will be returned.
offset_max	The maximum value that the `offset` query parameter can be. If it is higher then an HTTP 416 “Requested Range Not Satisfiable” response code will be returned with message “Error: offset value greater than maximum allowed.” If the value is “n/a” then the `offset` parameter is not allowed for this API key, and similar 416 error will be generated.
burst_size	The maximum number of API lookups that may be performed within this burst_window number of seconds.
burst_window	The number of seconds over which a burst of queries is measured.

Examples

Time-Based Quota Example

For a time-based quota, the following is an example of a /dnsdb/v2/rate_limit response performed on June 10, 2015 that indicates that the API key’s quota limit will be reset at midnight UTC on June 11, 2015, and that 999 lookups are remaining out of a total quota of 1000 lookups:

curl -H "Accept: application/x-ndjson" -H "X-API-Key: $DNSDB_API_KEY" \
  "https://api.dnsdb.info/dnsdb/v2/rate_limit"

Response:

{"rate": {"reset": 1433980800, "limit": 1000, "remaining": 999}}

Block-Based Quota Example

For a block-based quota, the following example indicates that the API key’s quota limit will expire at Mon Apr 15 19:28:34 2019, 592 lookups were used out of a total quota of 600 lookups, the burst size is 10 queries in 5 minutes, the maximum number of results that can be requested from a single query are 256, and offset will fail if higher than 3000000:

{"rate": {"reset": "n/a", "burst_size": 10, "expires": 1555370914, "burst_window": 300,
    "offset_max": 3000000, "results_max": 256, "limit": 600, "remaining": 8}}

Unlimited Quota Example

An unlimited quota API key has the corresponding /dnsdb/v2/rate_limit response:

{"rate": {"reset": "n/a", "limit": "unlimited", "remaining": "n/a"}}

X-RateLimit Headers

Responses from the data-fetching endpoints contain information in the HTTP response headers that are a subset of that obtained from the /dnsdb/v2/rate_limit endpoint. These values are embedded as the X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset, and for block-based quotas, X-RateLimit-Expires headers.

Time-Based Quota Headers

For a time-based quota, responses to lookups will contain response headers like:

X-RateLimit-Limit: 1000
X-RateLimit-Remaining: 999
X-RateLimit-Reset: 1433980800

Block-Based Quota Headers

For a block-based quota, responses to lookups will contain response headers like:

X-RateLimit-Limit: 600
X-RateLimit-Remaining: 8
X-RateLimit-Reset: n/a
X-RateLimit-Expires: 1555370914

Unlimited Quota Headers

For an unlimited API key, responses to lookups will contain response headers like:

X-RateLimit-Limit: unlimited
X-RateLimit-Remaining: n/a
X-RateLimit-Reset: n/a

Quota Exceeded Errors

If you have exceeded your quota, the HTTP 429 “Too Many Requests” response code will be generated with message ‘Error: Rate limit exceeded’.

For time-based quotas: The API key’s daily quota limit is exceeded. The quota will automatically replenish, usually at the start of the next day.
For block-based quotas: The block quota is exhausted. You may need to purchase a larger quota.
For burst rate secondary quotas: There were too many queries within the burst window. The window will automatically reopen at its end.