This document explains how to generate, view, and manage data insights for your structured data. Using AI-powered data insights helps you accelerate data exploration by automatically generating descriptions, relationship graphs, and SQL queries from your table and dataset metadata.
In BigQuery Studio, you can generate data insights for BigQuery datasets, tables, views, Google Cloud BigLake tables, BigQuery external tables, and Apache Iceberg namespaces.
In Knowledge Catalog, you can generate data insights for Apache Iceberg, Apache Hive, and SAP BDC tables and namespaces managed by Google Cloud's Lakehouse for Apache Iceberg.
Before you begin
Before using data insights, ensure you have completed the following prerequisites:
Required roles
To get the permissions that you need to use data insights, ask your administrator to grant you the following IAM roles:
-
Get read-only access to the generated insights:
Dataplex DataScan DataViewer (
roles/dataplex.dataScanDataViewer) on project containing the resource -
Read Apache Iceberg, Apache Hive, or SAP BDC table data:
BigLake Viewer (
roles/biglake.viewer) on resource -
Publish descriptions as aspects:
Dataplex Catalog Editor (
roles/dataplex.catalogEditor) on resource -
Publish queries as aspects:
Dataplex Entry and EntryLink Owner (
roles/dataplex.entryOwner) on resource
For more information about granting roles, see Manage access to projects, folders, and organizations.
These predefined roles contain the permissions required to use data insights. To see the exact permissions that are required, expand the Required permissions section:
Required permissions
The following permissions are required to use data insights:
-
dataplex.datascans.create -
dataplex.datascans.get -
dataplex.datascans.getData -
dataplex.datascans.run
You might also be able to get these permissions with custom roles or other predefined roles.
Enable APIs
To use data insights, enable the following APIs in your project:
- Dataplex API
- BigQuery API
- Gemini for Google Cloud API
Roles required to enable APIs
To enable APIs, you need the serviceusage.services.enable permission. If you
created the project, then you likely already have this permission through the
Owner role (roles/owner). Otherwise, you can get this permission through the
Service Usage Admin role (roles/serviceusage.serviceUsageAdmin).
Learn how to grant roles.
For more information about enabling the Gemini for Google Cloud API, see Enable the Gemini for Google Cloud API in a Google Cloud project.
Prepare data
For Google Cloud Lakehouse tables, ensure that your data is in Cloud Storage and you have created a Google Cloud Lakehouse table.
For Apache Iceberg REST Catalog tables, ensure your tables are registered in the Lakehouse runtime catalog.
Generate insights in BigQuery
Data insights for BigQuery datasets, tables, views, Google Cloud Lakehouse tables, and BigQuery external tables are generated using Gemini in BigQuery and can only be generated in BigQuery Studio.
You must first set up Gemini in BigQuery, then generate insights. After you generate insights, you can view and modify them in Knowledge Catalog.
For more information about generating insights in BigQuery, see the following documents:
Generate insights for Apache Iceberg tables and namespaces
In the Google Cloud console, go to the Knowledge Catalog Search page.
In the Filters, locate your asset type:
- For Apache Iceberg tables: Select Lakehouse.
- For Apache Iceberg namespaces: Set the
systemfilter to BIGLAKE and thetypefilter to namespace.
Select the Apache Iceberg table or namespace from the search results to open its entry details page.
Click the Insights tab. If the tab is empty, it means that the insights for this table aren't generated yet.
Choose a generation option:
To generate and permanently attach insights to the asset as metadata aspects, click Generate and publish. This makes the insights indexable, searchable, and visible to other users in your organization within the Knowledge Catalog.
To generate and view insights temporarily during your current session, click Generate without publishing.
For more information about the differences between the Generate and publish and