Skip to main content

Export Data to Google BigQuery

The Mondoo BigQuery integration lets you continuously export your space data, such as assets and vulnerabilities, to a Google BigQuery dataset. The export runs approximately every 24 hours.

Requirements​

  • The GCP IAM API enabled

  • The GCP BigQuery API enabled

  • The GCP CLI installed

  • Editor or Owner access to the space from which you want to export data

Create a service account for your Google BigQuery integration​

To access the data it needs, your BigQuery integration needs a GCP service account. To learn about service accounts, read Understanding service accounts in the Google documentation.

  1. Create a new GCP service account for the Mondoo integration to use.

    For instructions, read Creating and managing service accounts in the Google documentation.

    Note the email address created for the new service account.

  2. Create a JSON key for the service account.

    For instructions, read Create and manage service account keys in the Google documentation.

    Save the JSON file that downloads to your workstation when you create the key. You need it to configure the integration (in the next section below).

Create a BigQuery dataset​

Your BigQuery integration needs a GCP BigQuery dataset to which to export data. To learn about BigQuery datasets, read Introduction to datasets in the Google documentation.

  1. Create a new GCP BigQuery dataset for the Mondoo integration to use.

    For instructions, read Creating datasets in the Google documentation.

  2. Assign the "BigQuery Data Editor" role to the GCP service account you created in the instructions above.

    For instructions, read Grant access to a dataset in the Google documentation.

    For a description of the permissions that the "BigQuery Data Editor" role grants, read Understanding roles in the Google documentation.

Add a new BigQuery integration​

  1. Access the Integrations > Add > BigQuery page in one of two ways:

    • New space setup: After creating a new Mondoo account or creating a new space, the initial setup guide welcomes you. Select BROWSE INTEGRATIONS and then select BigQuery.

      Welcome to Mondoo Page

    • INTEGRATIONS page: Navigate to INTEGRATIONS. Under Exports, select BigQuery.

      integration-create-image

  2. In the Choose an integration name box, enter a name for the integration. Make it a name that lets you easily recognize the GCP BigQuery dataset.

  3. Identify the BigQuery dataset to use for the export.

    In the Enter the Dataset ID box, enter your BigQuery dataset's resource ID. To learn how to retrieve this value, read Listing datasets in the Google documentation.

  4. Under Provide your Google service account config, upload the GCP service account's JSON key that you downloaded in the previous section:

    Drag the file and drop it in the Drag and drop your .json file here box.

    OR

    In the Drag and drop your .json file here box, select the cloud icon and choose the file to upload.

    integration-create-image

  5. To complete the integration, select the CREATE INTEGRATION button.

    Mondoo begins exporting information from your space. When this initial export completes, the integration status becomes ACTIVE. Mondoo performs an export approximately every 24 hours.

View your BigQuery integration​

  1. In the side navigation bar, under Integrations, select BigQuery.

  2. In the list of BigQuery integrations, select the integration you want to view.

    integration-view-image

Statuses​

The possible statuses for a BigQuery integration are:

  • active: The integration is active/healthy.
  • error: Mondoo detected an error during export.

Export data to BigQuery at any time​

Mondoo exports your data to BigQuery approximately every 24 hours. You can also manually export data at any time. Use the SCHEDULE NOW button to request an export right away. Mondoo begins the export as soon as possible. If Mondoo is processing a lot of data, there can be some delay until Mondoo has sufficient resources.

Remove an integration​

To remove an integration, select the trash can icon. A confirmation prompt displays. Once you confirm the deletion, Mondoo removes the configured integration and stops triggering exports for the BigQuery dataset.

info

Mondoo does not delete your GCP BigQuery dataset or existing exported data. You must delete the dataset manually.

Next steps​