Schedule Workflows

This tutorial shows you how to use Cloud Scheduler to automatically execute Workflows so that a workflow runs on a particular schedule, in this case every 5 minutes.

Objectives

  1. Create and deploy a workflow that can receive runtime arguments.
  2. Create a Cloud Scheduler job that triggers your workflow and executes it every 5 minutes, passing in JSON arguments in the correct format.

Costs

In this document, you use the following billable components of Google Cloud:

To generate a cost estimate based on your projected usage, use the pricing calculator.

New Google Cloud users might be eligible for a free trial.

Before you begin

  1. Set up your environment for Cloud Scheduler, including creating an App Engine app.

  2. Enable the Workflows API.

    Roles required to enable APIs

    To enable APIs, you need the serviceusage.services.enable permission. If you created the project, then you likely already have this permission through the Owner role (roles/owner). Otherwise, you can get this permission through the Service Usage Admin role (roles/serviceusage.serviceUsageAdmin). Learn how to grant roles.

    Enable the API

  3. Set the default location used in this tutorial: 
      gcloud config set workflows/location REGION
     Replace REGION with the supported Workflows location of your choice.
  4. Create a service account for Workflows to use; for example, sa-name. 
      gcloud iam service-accounts create sa-name
  5. Grant your service account the workflows.invoker role so that the account has permission to trigger your workflow: 
      gcloud projects add-iam-policy-binding PROJECT_ID \
      --member "serviceAccount:sa-name@PROJECT_ID.iam.gserviceaccount.com" \
      --role "roles/workflows.invoker"
    6. Replace PROJECT_ID with your Google Cloud project ID.
  6. Grant the logging.logWriter role to the service account. 
      gcloud projects add-iam-policy-binding PROJECT_ID \
      --member "serviceAccount:sa-name@PROJECT_ID.iam.gserviceaccount.com" \
      --role "roles/logging.logWriter"

Create a workflow that receives runtime arguments

A workflow definition is made up of a series of steps described using the Workflows syntax, which can be written in either YAML or JSON format. After creating a workflow, you deploy it to make it available for execution.

Console

  1. In the Google Cloud console, go to the Workflows page:

    Go to Workflows

  2. Click Create.

  3. Enter myFirstWorkflow as a name for your new workflow.

  4. Select us-east1 for the region.

  5. Select the service account you created previously.

  6. Click Next.

  7. In the workflow editor, enter the following definition for your workflow.

    YAML

    main:
  params: [args]
  steps:
    - step1:
        assign:
          - outputVar: ${"Hello, " + args.firstName + " " + args.lastName + "!"}
    - step2:
        return: ${outputVar}

    JSON

    {
  "main": {
    "params": [
      "args"
    ],
    "steps": [
      {
        "step1": {
          "assign": [
            {
              "outputVar": "${\"Hello \" + args.firstName + \" \" + args.lastName}"
            }
          ]
        }
      },
      {
        "step2": {
          "return": "${outputVar}"
        }
      }
    ]
  }
}

    This workflow returns a "Hello" greeting to a person whose first and last name you pass as runtime arguments.

  8. Click Deploy.

gcloud

  1. Open a terminal.

  2. Save the following workflow definition as a YAML or JSON file, such as myFirstWorkflow.yaml or myFirstWorkflow.json.

    YAML

    main:
  params: [args]
  steps:
    - step1:
        assign:
          - outputVar: ${"Hello, " + args.firstName + " " + args.lastName + "!"}
    - step2:
        return: ${outputVar}

    JSON

    {
  "main": {
    "params": [
      "args"
    ],
    "steps": [
      {
        "step1": {
          "assign": [
            {
              "outputVar": "${\"Hello \" + args.firstName + \" \" + args.lastName}"
            }
          ]
        }
      },
      {
        "step2": {
          "return": "${outputVar}"
        }
      }
    ]
  }
}

    This workflow returns a "Hello" greeting to a person whose first and last name you pass as runtime arguments.

  3. Deploy the workflow by entering the following command:

    gcloud workflows deploy myFirstWorkflow \
    --location=us-east1 \
    --source=myFirstWorkflow.yaml_OR_json \
    --service-account=sa-name@PROJECT_ID.iam.gserviceaccount.com

    Replace yaml_OR_json with yaml or json depending on the format of the Workflows definition file that you created previously.

Schedule the workflow

Create a Cloud Scheduler job that triggers your workflow, using the service account you previously created.

Console

  1. In the Google Cloud console, go to the Cloud Scheduler page:

    Go to Cloud Scheduler

  2. Click Create Job.

  3. Set the Name to my-workflow-job.

  4. For Region, select us-east1 (South Carolina).

  5. For Frequency, enter: 

    */5 * * * *
     This will execute the job every 5 minutes. The interval is defined using unix-cron format.

  6. For Timezone, select a country and timezone.

    For example, select United States and Los Angeles or Pacific Daylight Time (PDT).

  7. Click Continue.

  8. For Target type, select HTTP.

  9. For URL, enter: 

    https://workflowexecutions.googleapis.com/v1/projects/PROJECT_ID/locations/us-east1/workflows/myFirstWorkflow/executions

  10. Leave the HTTP method at the default of POST.

  11. Add the following two HTTP headers:

    • Name: Content-Type and Value: application/octet-stream
    • Name: User-Agent and Value: Google-Cloud-Scheduler

  12. For Body, enter: 

    {"argument": "{\"firstName\":\"Sherlock\", \"lastName\":\"Holmes\"}"}