You use Model Armor templates to configure the screening of prompts and responses for your AI applications. The templates provide customized filters and thresholds for multiple safety and security categories. This document explains how to create and manage Model Armor templates. For more information, see Model Armor templates.
Before you begin
Before you begin, complete the following tasks.
Obtain the required permissions
To get the permissions that
you need to manage Model Armor templates,
ask your administrator to grant you the
Model Armor Admin (roles/modelarmor.admin) IAM role on Model Armor templates.
For more information about granting roles, see Manage access to projects, folders, and organizations.
You might also be able to get the required permissions through custom roles or other predefined roles.
Enable APIs
You must enable the Model Armor API before you can use Model Armor.
Console
Enable the Model Armor API.
Roles required to enable APIs
To enable APIs, you need the
serviceusage.services.enablepermission. If you created the project, then you likely already have this permission through the Owner role (roles/owner). Otherwise, you can get this permission through the Service Usage Admin role (roles/serviceusage.serviceUsageAdmin). Learn how to grant roles.Select the project where you want to activate Model Armor.
gcloud
Before you begin, follow these steps using the Google Cloud CLI with the Model Armor API:
In the Google Cloud console, activate Cloud Shell.
At the bottom of the Google Cloud console, a Cloud Shell session starts and displays a command-line prompt. Cloud Shell is a shell environment with the Google Cloud CLI already installed and with values already set for your current project. It can take a few seconds for the session to initialize.
Set the API endpoint override using the gcloud CLI
This step is only required if you are using the gcloud CLI to enable the Model Armor API. You must manually set the API endpoint override to ensure the gcloud CLI correctly routes requests to the Model Armor service.
Run the following command to set the API endpoint for the Model Armor service.
gcloud config set api_endpoint_overrides/modelarmor "https://modelarmor.LOCATION.rep.googleapis.com/"
Replace LOCATION with the region where you want to use
Model Armor.
Create a Model Armor template
Model Armor templates define the specific filters and thresholds that Model Armor uses to screen prompts and responses for safety and security risks. To create a Model Armor template, follow these steps:
Console
In the Google Cloud console, go to the Model Armor page.
Verify that you are viewing the project that you activated Model Armor on.
On the Model Armor page, click Create Template. The Create Template page is displayed.
Specify the Template ID. The template ID can contain letters, digits, or hyphens. It cannot exceed 63 characters, contain spaces, or start with a hyphen.
Select a Region where the Model Armor templates will run. You cannot change the region later.
Optional: Add Labels. Labels are key-value pairs that you can use to group related templates.
In the Filter version section, select either the filter version number or the filter version alias.
Select modality to specify whether you want to screen text, images, or both.
In the Detections section, configure the detection settings.
Optional: If you select Sensitive Data Protection detection, you need to configure the Sensitive Data Protection settings.
In the Responsible AI section, set the confidence level for each content filter.
Optional: In the Configure logging section, select the operations that you want to configure logging for.
Optional: Select Enable multi-language support to use the multi-language detection settings.
Click Create.
gcloud
Run the following command:
gcloud model-armor templates create TEMPLATE_ID --project=PROJECT_ID --location=LOCATION \
--rai-settings-filters='[{ "filterType": "HATE_SPEECH", "confidenceLevel": "MEDIUM_AND_ABOVE" },{ "filterType": "HARASSMENT", "confidenceLevel": "MEDIUM_AND_ABOVE" },{ "filterType": "SEXUALLY_EXPLICIT", "confidenceLevel": "MEDIUM_AND_ABOVE" }]' \
--basic-config-filter-enforcement=enabled \
--pi-and-jailbreak-filter-settings-enforcement=enabled \
--pi-and-jailbreak-filter-settings-confidence-level=HIGH \
--malicious-uri-filter-settings-enforcement=enabled \
--template-metadata-custom-llm-response-safety-error-code=798 \
--template-metadata-custom-llm-response-safety-error-message="test template llm response evaluation failed" \
--template-metadata-custom-prompt-safety-error-code=799 \
--template-metadata-custom-prompt-safety-error-message="test template prompt evaluation failed" \
--template-metadata-ignore-partial-invocation-failures \
--template-metadata-log-operations \
--template-metadata-log-sanitize-operations
Replace the following:
LOCATION: the location of the template.TEMPLATE_ID: the ID of the template.PROJECT_ID: the ID of the project that the template belongs to.
REST
Use the following command to create a new Model Armor template.
curl -X POST \
-d "{'FILTER_CONFIG': {} }" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
"https://modelarmor.LOCATION.rep.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/templates?template_id=TEMPLATE_ID"
Replace the following:
FILTER_CONFIG: the filter configuration for the template.PROJECT_ID: the ID of the project that the template belongs to.TEMPLATE_ID: the ID of the template to be created.LOCATION: the location of the template.
The following example shows the Model Armor template
configuration. In this example, the Responsible AI filters are configured for hate speech, harassment, dangerous content, and sexually explicit content with varying
confidence levels.
The prompt injection and jailbreak detection filter is enabled with a HIGH confidence level, which means only content with a high likelihood will be flagged. The malicious URI filter is enabled.
export FILTER_CONFIG='{
"filterConfig": {
"raiSettings": {
"raiFilters": [{
"filterType": "HATE_SPEECH",
"confidenceLevel": "MEDIUM_AND_ABOVE"
}, {
"filterType": "HARASSMENT",
"confidenceLevel": "HIGH"
}, {
"filterType": "DANGEROUS",
"confidenceLevel": "MEDIUM_AND_ABOVE"
},{
"filterType": "SEXUALLY_EXPLICIT",
"confidenceLevel": "MEDIUM_AND_ABOVE"
}]
},
"piAndJailbreakFilterSettings": {
"filterEnforcement": "ENABLED",
"confidenceLevel": "HIGH"
},
"maliciousUriFilterSettings": {
"filterEnforcement": "ENABLED"
}
}
}'
curl -X POST \
-d "$FILTER_CONFIG" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
"https://modelarmor.LOCATION.rep.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION_ID/templates?template_id=TEMPLATE_ID"
To create a Model Armor template with multi-language detection
and modalities enabled, you need to pass the TEMPLATE_CONFIG parameter
in your command. This parameter defines the configuration for the language
detection and modalities settings.
Use the following command to create a new Model Armor template with multi-language detection enabled.
curl -X POST \
-d "{'TEMPLATE_CONFIG': {} }" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
"https://modelarmor.LOCATION.rep.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/templates?template_id=TEMPLATE_ID"
The following example shows the Model Armor template configuration with multi-language detection and modalities enabled.
export TEMPLATE_CONFIG='{
"filterConfig": {
"raiSettings": {
"raiFilters": [{
"filterType": "HATE_SPEECH",
"confidenceLevel": "MEDIUM_AND_ABOVE"
}, {
"filterType": "HARASSMENT",
"confidenceLevel": "HIGH"
}, {
"filterType": "DANGEROUS",
"confidenceLevel": "MEDIUM_AND_ABOVE"
},{
"filterType": "SEXUALLY_EXPLICIT",
"confidenceLevel": "MEDIUM_AND_ABOVE"
}]
},
"piAndJailbreakFilterSettings": {
"filterEnforcement": "ENABLED",
"confidenceLevel": "HIGH"
},
"maliciousUriFilterSettings": {
"filterEnforcement": "ENABLED"
}
},
"templateMetadata": {
"multiLanguageDetection": {
"enableMultiLanguageDetection": true
},
"modalities": ["MODALITY_IMAGE", "MODALITY_TEXT"]
}
}'
curl -X POST \
-d "$TEMPLATE_CONFIG" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
"https://modelarmor.LOCATION.rep.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION_ID/templates?template_id=TEMPLATE_ID"
C#
To run this code, first set up a C# development environment and install the Model Armor C# SDK.
Go
To run this code, first set up a Go development environment and install the Model Armor Go SDK.
Java
To run this code, first set up a Java development environment and install the Model Armor Java SDK.
Node.js
To run this code, first set up a Node.js development environment and install the Model Armor Node.js SDK.