Create a Confluent Cloud import topic

A Confluent Cloud import topic lets you continuously ingest data from Confluent Cloud as an external source and into Pub/Sub. Then you can stream the data into any of the destinations that Pub/Sub supports.

This document shows you how to create and manage Confluent Cloud import topics. To create a standard topic, see Create a standard topic.

For more information about import topics, see About import topics.

Before you begin

• Know more about the Pub/Sub publish process.

• Configure the required roles and permissions to manage Confluent Cloud import topics including the following:

  • Add the Pub/Sub publisher role to the Pub/Sub service account

  • Add the Pub/Sub service agent role to the Pub/Sub service account, if not already granted.

  • Add the service account user role to the service account

• Set up workload identity federation so that Google Cloud can access the external streaming service.

Required roles and permissions

To get the permissions that you need to create and manage Confluent Cloud import topics, ask your administrator to grant you the Pub/Sub Editor (roles/pubsub.editor) IAM role on your topic or project. For more information about granting roles, see Manage access to projects, folders, and organizations.

This predefined role contains the permissions required to create and manage Confluent Cloud import topics. To see the exact permissions that are required, expand the Required permissions section:

Required permissions

The following permissions are required to create and manage Confluent Cloud import topics:

• Create an import topic: pubsub.topics.create
• Delete an import topic: pubsub.topics.delete
• Get an import topic: pubsub.topics.get
• List an import topic: pubsub.topics.list
• Publish to an import topic: pubsub.topics.publish and pubsub.serviceAgent
• Update an import topic: pubsub.topics.update
• Get the IAM policy for an import topic: pubsub.topics.getIamPolicy
• Configure the IAM policy for an import topic: pubsub.topics.setIamPolicy

You might also be able to get these permissions with custom roles or other predefined roles.

You can configure access control at the project level and the individual resource level.

Set up federated identity to access Confluent Cloud

Workload Identity Federation lets Google Cloud services access workloads running outside of Google Cloud. With identity federation, you don't need to maintain or pass credentials to Google Cloud to access your resources in other clouds. Instead, you can use the identities of the workloads themselves to authenticate to Google Cloud and access resources.

Create a service account in Google Cloud

This is an optional step. If you already have a service account, you can use it in this procedure instead of creating a new service account. If you are using an existing service account, go to Record the service account unique ID for the next step.

For Confluent Cloud import topics, Pub/Sub uses the service account as the identity to access resources from Confluent Cloud.

For more information about creating a service account, including prerequisites, required roles and permissions, and naming guidelines, see Create service accounts. After you create a service account, you might need to wait for 60 seconds or more before you use the service account. This behavior occurs because read operations are eventually consistent; it can take time for the new service account to become visible.

Record the service account unique ID

You need a service account unique ID to set up the identity provider and pool in the Confluent Cloud console.

1. In the Google Cloud console, go to the Service account details page.

   Go to service account

2. Click the service account that you just created or the one that you are planning to use.

3. From the Service account details page, record the Unique ID number.

   You need the ID as part of the workflow to set up the identity provider and pool in the Confluent Cloud console.

Add the service account token creator role to the Pub/Sub service account

The Service account token creator role (roles/iam.serviceAccountTokenCreator) lets principals create short-lived credentials for a service account. These tokens or credentials are used to impersonate the service account.

For more information about service account impersonation, see Service account impersonation.

You can also add the Pub/Sub publisher role (