Skip to content

Chronicle integration troubleshooting

This guide covers limitations, known issues, troubleshooting steps, cost information, and references for the Chronicle App for DomainTools.

Limitations

  • Chronicle Backstory Notation (CBN) parser will only be able to parse the DomainTools events.
  • We suggest using the second generation of Cloud Function. The first generation of Cloud Function has a maximum execution time of 9 minutes and the second generation of Cloud Function has a maximum execution time of 60 minutes. If the execution time of the Cloud Function exceeds timeout then there are chances that the complete data (Alerts, Activities, Devices, Vulnerabilities) isn't ingested in the Chronicle.
  • Chronicle search API calls have a rate limit of 120 queries per hour, with each query yielding a maximum of 10,000 results. Hence, in total, we can fetch 1200k Unified Data Model (UDM) data from the Chronicle in an hour. [More details can be found at https://cloud.google.com/chronicle/docs/reference/search-api#search_api_query_limits
  • The rate limit for DomainTools enrich API is 60/minute. Hence in total, we could fetch around 360k domains in an hour.
  • The cloud scheduler runs a maximum of 30 minutes. If cloud function execution is taking more than 30 minutes, execution of the cloud function will be stopped.

Known issues

  • The looker dashboard doesn't display data in the drill down table when there are too many records to be displayed.
  • In the Enrichment Explorer dashboard, the Threat Type will always populate with Not a Threat filter value if there is a single record present in the applied time range filter.
  • Looker will only show data from the past 180 days, but this can vary as per the retention policy configured in BigQuery.
  • According to the query time zone selected by the user in connection with the Chronicle database, the Looker dashboards would be reflected according to the configured timezone.
  • While redirecting to Google Chronicle, it will show an error like this 'Search has encountered an error and couldn't load data. Please try again, and contact support if this error continues.' if the searched date range is more than 3 months.

Troubleshooting

This section describes common issues that might occur during deployment or operation of the app and steps to resolve them.

Monitoring and logs

Use GCloud logs for troubleshooting. To access logs:

  1. Log in to https://console.cloud.google.com/ using valid credentials.
  2. Navigate to Cloud functions and click the deployed function.
  3. Open the logs module.
  4. Filter logs by severity as needed.

Missing default logs: GCloud default logs may not appear in the logs module after testing the function manually or when the scheduler job invokes the function. Refresh the GCloud page to resolve this issue.

Deployment and configuration

Testing timing: If you test the cloud function immediately after deploying it, the function may not work as expected. Wait a few seconds before testing.

Database connection: If data isn't displayed on the dashboard, verify that the database connection is configured correctly.

Cloud Function execution

Memory limit exceeded: If the cloud function stops execution because memory exceeds the limit, reconfigure the cloud function's memory configuration and increase the memory limit.

Looker dashboard issues

Cached data: The Looker dashboard may display cached data instead of the latest events. To resolve:

  1. Click the three dots on the rightmost side of the dashboard.
  2. Click Clear cache and refresh.

Data not displaying: If data isn't displayed on the dashboard, verify that:

  • The database connection is configured correctly
  • Filters are not too restrictive

Incorrect filters: If desired events aren't showing in the visualization, ensure that the filters in the dashboard are configured correctly. Overly restrictive filters may prevent the dashboard from displaying data.

Slow or unresponsive dashboard: The dashboard may be slow to load or unresponsive due to:

  • Data source being unavailable
  • Too much data being queried
  • Query performance issues
  • Rendering problems

Enrichment Explorer domain filter mismatch: In the Enrichment Explorer dashboard, data in the table and domains in the Domain filter may not match. This occurs because the Domain filter populates with domains from the Last Enriched filter in ascending alphabetical order, while the table displays data in descending order of the Last Enriched DateTime column.

To display domain details that aren't in the table but are within the Last Enriched time range:

  1. Search for the domain in the Domain filter.
  2. Select the domain to populate the dashboard.

Chronicle integration

Redirect errors: If you encounter an error while redirecting to Google Chronicle, check the searched date range. Chronicle supports a maximum of 3 months for search ranges.

Data validation: If you're unable to see data in the dashboards, verify that data exists in Chronicle:

  1. Open your Chronicle instance.
  2. Click the Application Menu in the upper right corner, then select UDM Search.
  1. In the UDM Search Bar, enter this query:
  2. metadata.log_type = "DOMAINTOOLS_THREATINTEL"
  3. Ensure the date range matches the dashboard and all other filters are unselected.
  4. If no data appears in Chronicle, Looker dashboards won't populate. If data appears, verify tile-specific data in Chronicle for the same time range.

Google Cloud Platform resources and services approximate cost details

The following Google Cloud services are required to run the Chronicle App for DomainTools. Costs are approximate and based on standard configurations.

Memorystore for Redis

Caches domains based on the Time To Live (TTL) value you configure.

  • Service tier: Standard
  • Instance size: 4 GiB
  • Approximate cost: ~$185/month

For pricing details, see Memorystore for Redis pricing.

Cloud Functions

Pulls data from Chronicle and enriches it through the DomainTools API.

  • Memory: 8192 MB
  • CPU: 4.8 GHz
  • Execution time per function: 3600 ms
  • Invocations per month: 1500
  • Minimum instances: 1
  • Approximate cost: ~$66/month

For pricing details, see Cloud Functions pricing.

Cloud Storage

Storage bucket that manages API checkpoints.

  • Total storage: 1 GiB
  • Approximate cost: ~$0.02/month

For pricing details, see Cloud Storage pricing.

Secret Manager

Stores and maintains credentials.

  • Active secret versions per replica location: 4
  • Access operations: 1500
  • Approximate cost: ~$0/month

For pricing details, see Secret Manager pricing.

Cloud Scheduler

Executes the cloud function at a specific time interval.

  • Total jobs: 1
  • Approximate cost: ~$0/month

For pricing details, see Cloud Scheduler pricing.

Serverless VPC access

Connects directly to your Virtual Private Cloud (VPC) network from serverless environments such as Cloud Run, App Engine, or Cloud Functions.

For pricing details, see VPC pricing.

Looker

Visualization tool that displays enriched events from Chronicle. Cost depends on the platform edition you select.

For pricing details, see Looker pricing.

You can estimate the total price of these Google Cloud services with the pricing calculator.

References