Configure exports

This page explains how to export traces by using the Cloud Trace API and the Google Cloud CLI. You must use version 274.0.0 or later of the Google Cloud CLI. For information on how to update the Google Cloud CLI, see gcloud components update .

Some samples on this page were generated using curl. For information on configuring this tool, see Using curl.

For an example that illustrates using the Google Cloud CLI commands to list, create, describe, update, and delete a sink, see End-to-end example.

Terminology

To simplify the examples on this page, environment variables have been used.

The Google Cloud CLI examples use the following environment variables:

SINK_ID: The name, or the identifier, of the sink. For example, my-sink. You don't need to provide the fully qualified command to the Google Cloud CLI, as it can determine your Google Cloud project.
DESTINATION: Stores the fully qualified name of the destination. This must be a BigQuery dataset. For example, a valid destination is:
```
bigquery.googleapis.com/projects/DESTINATION_PROJECT_NUMBER/datasets/DATASET_ID
```
where DESTINATION_PROJECT_NUMBER is the Google Cloud project number of the destination, and DATASET_ID is the BigQuery dataset identifier.

Note: There is limited semantic checking on the format of the destination. For example, if you omit the DATASET_ID when you issue a create command, the create command doesn't fail but traces won't be exported successfully.

The curl examples use the following environment variables:

ACCESS_TOKEN: Stores the authorization token. For more information, see Using curl.
PROJECT_ID: Stores the Google Cloud project identifier or project number.
PROJECT_NUMBER: Stores the Google Cloud project number.
SINK_ID: The name, or the identifier, of the sink. For example, my-sink.
SINK_BODY: Stores the description of a TraceSink resource. The TraceSink resource includes a name and the sink destination. The name must specify the Google Cloud project number.
DESTINATION: Stores the fully qualified name of the destination. This must be a BigQuery dataset.

Configuring the destination

To export traces to BigQuery, do the following:

Create the destination dataset.
Create the sink by using the Cloud Trace API or Google Cloud CLI. For details, see Creating a sink.
Grant the sink the role of dataEditor for your BigQuery dataset:
1. Obtain the writer identity from the sink. For information on writer identity, see Sink properties and terminology.
  
  The writer identity for a sink is included in the response data to the create command. It is also included in the response data of the list command.
2. Add the sink's writer identity as a service account to your BigQuery dataset and give it the role of BigQuery data editor.
  - To add the permissions using Google Cloud console, see Controlling access to a dataset.
  - To add the permissions using Google Cloud CLI, use the add-iam-policy-binding command and supply your Google Cloud project identifier and the sink's writer identity:
```
gcloud projects add-iam-policy-binding ${DESTINATION_PROJECT_ID} \
  --member serviceAccount:${WRITER_IDENTITY} \
  --role roles/bigquery.dataEditor
```
    In previous command, WRITER_IDENTITY is an environment variable that stores the writer identity of the sink and DESTINATION_PROJECT_ID is the Google Cloud project identifier of the BigQuery dataset.

Listing sinks

To list all sinks in your Google Cloud project, including their writer identities, invoke the traceSinks.list method.

gcloud

To list the sinks defined with the default project by using the Google Cloud CLI, run the following command:

gcloud alpha trace sinks list

Protocol

To list sinks by using curl, send a GET request to:

https://cloudtrace.googleapis.com/v2beta1/projects/${PROJECT_NUMBER}/traceSinks

For example, the following request retrieves all sinks:

curl --http1.1 --header "Authorization: Bearer ${ACCESS_TOKEN}"  https://cloudtrace.googleapis.com/v2beta1/projects/${PROJECT_NUMBER}/traceSinks

Show details of a specific sink

To show the details of a specific sink that is in your Google Cloud project, invoke the traceSinks.get method.

gcloud

To display the details of the sink whose identifier is stored in SINK_ID by using the Google Cloud CLI, run the following command:

gcloud alpha trace sinks describe ${SINK_ID}

Protocol

To display the details of the sink whose identifier is stored in SINK_ID by using curl, send a GET request to:

https://cloudtrace.googleapis.com/v2beta1/projects/${PROJECT_NUMBER}/traceSinks/%{SINK_ID}

For example, the following request retrieves the details of the specified sink:

curl --http1.1 --header "Authorization: Bearer ${ACCESS_TOKEN}"  https://cloudtrace.googleapis.com/v2beta1/projects/${PROJECT_NUMBER}/traceSinks/${SINK_ID}

Creating a sink

To create a sink in your Google Cloud project, invoke the traceSinks.create method. The destination for a sink must be a BigQuery dataset.

This dataset must exist before you create the sink. Trace doesn't verify the existence of the destination. See Creating datasets for information on creating BigQuery datasets.

gcloud

To create a sink by using the Google Cloud CLI, run the following command:

gcloud alpha trace sinks create ${SINK_ID} ${DESTINATION}

Protocol

To create a sink by using curl, send a POST request to:

https://cloudtrace.googleapis.com/v2beta1/projects/${PROJECT_NUMBER}/traceSinks

For example, to create a sink named test_sink to export trace spans to test_dataset in the project with ${PROJECT_NUMBER}, define the environment variable SINK_BODY as shown:

SINK_BODY='{"name":"projects/12345/traceSinks/test_sink","output_config":{"destination":"bigquery.googleapis.com/projects/12345/datasets/test_dataset"}}'

To create the sink, run the following command:

curl --http1.1 --header "Authorization: Bearer ${ACCESS_TOKEN}"  --header "Content-Type: application/json"  -X POST -d ${SINK_BODY} https://cloudtrace.googleapis.com/v2beta1/projects/${PROJECT_NUMBER}/traceSinks

It might take several minutes after you create a sink before trace spans are exported to the destination.

Deleting a sink

To delete a sink that is in your Google Cloud project, invoke the traceSinks.delete command.

gcloud

To delete the sink whose identifier is stored in SINK_ID by using the Google Cloud CLI, run the following command:

gcloud alpha trace sinks delete ${SINK_ID}

Protocol

To delete the sink whose identifier is stored in SINK_ID by using curl, send a DELETE request to:

https://cloudtrace.googleapis.com/v2beta1/projects/${PROJECT_NUMBER}/traceSinks/${SINK_ID}

For example, the following request deletes a sink:

curl --http1.1 --header "Authorization: Bearer ${ACCESS_TOKEN}"  -X DELETE https://cloudtrace.googleapis.com/v2beta1/projects/${PROJECT_NUMBER}/traceSinks/${SINK_ID}

Updating a sink

To update a sink that is in your Google Cloud project, invoke the traceSinks.patch command.

This dataset must exist before you create the sink. Trace doesn't verify the existence of the destination.

gcloud

To update the sink whose identifier is stored in SINK_ID by using the Google Cloud CLI, run the following command:

gcloud alpha trace sinks update ${SINK_ID} ${DESTINATION}

The environment variable DESTINATION, stores the new destination for the sink.

Protocol

To update the destination of the sink whose identifier is stored in SINK_ID by using curl, send a PATCH request to:

https://cloudtrace.googleapis.com/v2beta1/projects/${PROJECT_NUMBER}/traceSinks/${SINK_ID}

For example, the following request updates the destination of a sink:

curl --http1.1 --header "Authorization: Bearer ${ACCESS_TOKEN}"  --header "Content-Type: application/json"  -X PATCH -d ${SINK_BODY} https://cloudtrace.googleapis.com/v2beta1/projects/${PROJECT_NUMBER}/traceSinks/${SINK_ID}?update_mask=output_config.destination

For an example of a SINK_BODY, see the example to create a sink.

It might take several minutes after you update a sink before trace spans are exported to the new destination.

End-to-end example

This section illustrates using the Google Cloud CLI commands to list, create, describe, update, and delete a sink. The commands were executed for a project with the project identifier a-sample-project. This project was pre-configured to contain 2 BigQuery datasets. Lastly, in these examples, the sink and the destination are in the same project. This isn't a requirement. The sink and destination can be in different Google Cloud projects.

Configuration steps

Verify the default project setting:

$ gcloud config list

Sample response:

[compute]
zone = us-east1-b
[core]
account = user@example.com
disable_usage_reporting = True
project = a-sample-project

Your active configuration is: [default]

Verify that Google Cloud CLI is at least 274.0.0:

$ gcloud --version

Sample response:

Google Cloud SDK 275.0.0
alpha 2020.01.03
beta 2020.01.03
bq 2.0.51
core 2020.01.03
gsutil 4.46
kubectl 2020.01.03

Identify available datasets in the default project:
```
$ bq ls
```
Sample response:
```
    datasetId
 ---------------
  dataset_1
  dataset_other
```
Note that the first time you use bq commands, you might need to select the project.

Setup the environment variables

Set the env variables used by the Google Cloud CLI command:
```
$ PROJECT_ID=a-sample-project
$ PROJECT_NUMBER=123456789000
$ SINK_ID=a-sample-sink
$ DATA_SET_NAME=dataset_1
$ DESTINATION=bigquery.googleapis.com/projects/${PROJECT_NUMBER}/datasets/${DATA_SET_NAME}
```
In this example, the destination and sink are in the same project. This isn't a requirement. The sink and destination can be in different Google Cloud projects.

Verify the settings:

$ echo $SINK_ID
a-sample-sink
$ echo $DATA_SET_NAME
dataset_1
$ echo $DESTINATION
bigquery.googleapis.com/projects/123456789000/datasets/dataset_1

List sinks

To list all sinks, run the following command:

$ gcloud alpha trace sinks list

Because this project doesn't have any sinks, the response is:

Listed 0 items.

Create a sink

To create a sink, run the following command:
```
$ gcloud alpha trace sinks create ${SINK_ID} ${DESTINATION}
```
Sample response:
```
You can give permission to the service account by running the following command.
gcloud projects add-iam-policy-binding bigquery-project \
--member serviceAccount:export-0000001cbe991a08-3434@gcp-sa-cloud-trace.iam.gserviceaccount.com \
--role roles/bigquery.dataEditor
```
Notice that the service account name includes export-0000001cbe991a08-3434. The number 0000001cbe991a08 is the hexadecimal representation of the PROJECT_NUMBER. The value 3434 is a random value.

Prior to executing the Google Cloud CLI in the previous response, you must replace bigquery-project with your project identifier.

Verify sink was created:

$ gcloud alpha trace sinks list

Sample response:

NAME           DESTINATION                                                       WRITER_IDENTITY
a-sample-sink  bigquery.googleapis.com/projects/123456789000/datasets/dataset_1  export-0000001cbe991a08-3434@gcp-sa-cloud-trace.iam.gserviceaccount.com

Describe the sink in detail:

$ gcloud alpha trace sinks describe ${SINK_ID}

Sample response:

destination: bigquery.googleapis.com/projects/123456789000/datasets/dataset_1
name: a-sample-sink
writer_identity: export-0000001cbe991a08-3434@gcp-sa-cloud-trace.iam.gserviceaccount.com

Grant the bigquery.dataEditor permission to the write identity of the sink. The sink create command returns a Google Cloud CLI command that you can use to update the permission.

For this command to be successful, do the following:
- Ensure that you have permission to modify the destination permissions.
- Replace bigquery-project with your project identifier:
```
gcloud projects
```

Configure exports Stay organized with collections Save and categorize content based on your preferences.

Terminology

Configuring the destination

Listing sinks

gcloud

Protocol

Show details of a specific sink

gcloud

Protocol

Creating a sink

gcloud

Protocol

Deleting a sink

gcloud

Protocol

Updating a sink

gcloud

Protocol

End-to-end example

Configuration steps

Setup the environment variables

List sinks

Create a sink

Configure exports