Customizing match likelihood

Using hotword rules, you can further extend built-in and custom infoType detectors with powerful context rules. A hotword rule instructs Sensitive Data Protection to adjust the likelihood of a finding, depending on whether a hotword occurs near that finding. A hotword rule is a kind of inspection rule, which is specified in rule sets. Each rule is applied to a set of built-in or custom infoTypes.

Anatomy of a hotword rule

An infoType detector can have zero or more hotword rules. In your inspection configuration, you define each HotwordRule object inside the rules array, as follows:

"rules":[
  {
    "hotwordRule":{
      "hotwordRegex":{
        "pattern":"REGEX_PATTERN"
      },
      "proximity":{
        "windowAfter":"NUM_CHARS_TO_CONSIDER_AFTER_FINDING",
        "windowBefore":"NUM_CHARS_TO_CONSIDER_BEFORE_FINDING"
      }
      "likelihoodAdjustment":{
        "fixedLikelihood":"LIKELIHOOD_VALUE"
             -- OR --
        "relativeLikelihood":"LIKELIHOOD_ADJUSTMENT"
      },
    }
  },
  ...
]

Replace the following:

  • REGEX_PATTERN: a regular expression (Regex object) that defines what qualifies as a hotword.
  • NUM_CHARS_TO_CONSIDER_AFTER_FINDING: a range of characters after the finding. Sensitive Data Protection analyzes this range to determine whether a hotword occurs near the finding.

  • NUM_CHARS_TO_CONSIDER_BEFORE_FINDING: a range of characters before the finding. Sensitive Data Protection analyzes this range to determine whether a hotword occurs near the finding.

  • LIKELIHOOD_VALUE: a fixed Likelihood level to set the finding to.

  • LIKELIHOOD_ADJUSTMENT: a number that indicates how much Sensitive Data Protection must increase or decrease the likelihood of the finding. A positive integer increases the likelihood level, and a negative integer decreases it. For example, if a finding would be POSSIBLE without the detection rule and relativeLikelihood is 1, then the finding is upgraded to LIKELY. If relativeLikelihood is -1, then the finding is downgraded to UNLIKELY. Likelihood can never drop lower than VERY_UNLIKELY or exceed VERY_LIKELY. In these cases, the likelihood level remains the same. For example, if the base likelihood is VERY_LIKELY and the relativeLikelihood is 1, the final likelihood remains to be VERY_LIKELY.

Hotword example: Match medical record numbers

Suppose you want to detect a custom infoType such as a medical record number (MRN) in the form "###-#-#####". Also, you want Sensitive Data Protection to increase the match likelihood of each finding that follows the hotword "MRN".

Example values:

  • 123-4-56789 would match as POSSIBLE.
  • MRN 123-4-56789 would match as VERY_LIKELY.

The following JSON example and code snippets show you how to configure the hotword rule. This example uses a custom regular expression detector.

In this example, note the following:

  • The request defines the C_MRN custom infoType, which is a detector for any string that matches the regular expression [0-9]{3}-[0-9]{1}-[0-9]{5}.
  • The regular expression (?i)(mrn|medical)(?-i) defines the hotword. Sensitive Data Protection searches for this hotword within the range of characters defined in the proximity field.
  • For each C_MRN finding that has a hotword within the set proximity, Sensitive Data Protection sets the likelihood level to VERY_LIKELY.

C#

To learn how to install and use the client library for Sensitive Data Protection, see Sensitive Data Protection client libraries.

To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, see Set up authentication for a local development environment. 


using System;
using Google.Api.Gax.ResourceNames;
using Google.Cloud.Dlp.V2;
using static Google.Cloud.Dlp.V2.CustomInfoType.Types;

public class InspectDataWithHotwordRule
{
    public static InspectContentResponse InspectDataHotwordRule(
        string projectId,
        string text,
        string customRegex,
        string hotwordRegex,
        InfoType infoType = null)
    {
        // Instantiate dlp client.
        var dlp = DlpServiceClient.Create();

        // Construct the content item.
        var contentItem = new ContentItem
        {
            ByteItem = new ByteContentItem
            {
                Type = ByteContentItem.Types.BytesType.TextUtf8,
                Data = Google.Protobuf.ByteString.CopyFromUtf8(text)
            }
        };

        // Construct the info type if null.
        var infotype = infoType ?? new InfoType { Name = "C_MRN" };

        // Construct the custom regex detector.
        var customInfoType = new CustomInfoType
        {
            InfoType = infotype,
            Regex = new Regex { Pattern = customRegex },
            Likelihood = Likelihood.Possible
        };

        // Construct hotword rule.
        var hotwordRule = new DetectionRule.Types.HotwordRule
        {
            HotwordRegex = new Regex { Pattern = hotwordRegex },
            LikelihoodAdjustment = new DetectionRule.Types.LikelihoodAdjustment
            {
                FixedLikelihood = Likelihood.VeryLikely
            },
            Proximity = new DetectionRule.Types.Proximity
            {
                WindowBefore = 10
            }
        };

        // Construct the rule set for the inspect config.
        var inspectionRuleSet = new InspectionRuleSet
        {
            InfoTypes = { infotype },
            Rules =
            {
                new InspectionRule
                {
                    HotwordRule = hotwordRule
                }
            }
        };

        // Construct the inspect config.
        var inspectConfig = new InspectConfig
        {
            CustomInfoTypes = { customInfoType },
            IncludeQuote = true,
            RuleSet = { inspectionRuleSet },
        };

        // Construct the request.
        var request = new InspectContentRequest
        {
            ParentAsLocationName = new LocationName(projectId, "global"),
            Item = contentItem,
            InspectConfig = inspectConfig
        };

        // Call the API.
        var response = dlp.InspectContent(request);

        // Inspect the response.
        Console.WriteLine($"Findings: {response.Result.Findings.Count}");
        foreach (var f in response.Result.Findings)
        {
            Console.WriteLine("Quote: " + f.Quote);
            Console.WriteLine("Info type: " + f.InfoType.Name);
            Console.WriteLine("Likelihood: " + f.Likelihood);
        }
        return response;
    }
}

Go

To learn how to install and use the client library for Sensitive Data Protection, see Sensitive Data Protection client libraries.

To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, see Set up authentication for a local development environment. 

import (
	"context"
	"fmt"
	"io"

	dlp "cloud.google.com/go/dlp/apiv2"
	"cloud.google.com/go/dlp/apiv2/dlppb"
)

// inspectWithHotWordRules inspects data with hot word rule, it uses custom
// regex with a hot word rule to increase the likelihood match
func inspectWithHotWordRules(w io.Writer, projectID, textToInspect string) error {
	// projectID := "my-project-id"
	// textToInspect := "Patient's MRN 444-5-22222 and just a number 333-2-33333"

	ctx := context.Background()

	// Initialize a client once and reuse it to send multiple requests. Clients
	// are safe to use across goroutines. When the client is no longer needed,
	// call the Close method to cleanup its resources.
	client, err := dlp.NewClient(ctx)
	if err != nil {
		return err
	}

	// Closing the client safely cleans up background resources.
	defer client.Close()

	// Specify the type and content to be inspected.
	contentItem := &dlppb.ContentItem{
		DataItem: &dlppb.ContentItem_ByteItem{
			ByteItem: &dlppb.ByteContentItem{
				Type: dlppb.ByteContentItem_TEXT_UTF8,
				Data: []byte(textToInspect),
			},
		},
	}