This topic covers the available de-identification techniques, or transformations, in Sensitive Data Protection.
Types of de-identification techniques
Choosing the de-identification transformation you want to use depends on the kind of data you want to de-identify and for what purpose you're de-identifying the data. The de-identification techniques that Sensitive Data Protection supports fall into the following general categories:
- Redaction: Deletes all or part of a detected sensitive value.
- Replacement: Replaces a detected sensitive value with a specified surrogate value.
- Masking: Replaces a number of characters of a sensitive value with a specified surrogate character, such as a hash (#) or asterisk (*).
- Crypto-based tokenization: Encrypts the original sensitive data value using a cryptographic key. Sensitive Data Protection supports several types of tokenization, including transformations that can be reversed, or "re-identified."
- Bucketing: "Generalizes" a sensitive value by replacing it with a range of values. (For example, replacing a specific age with an age range, or temperatures with ranges corresponding to "Hot," "Medium," and "Cold.")
- Date shifting: Shifts sensitive date values by a random amount of time.
- Time extraction: Extracts or preserves specified portions of date and time values.
The remainder of this topic covers each different type of de-identification transformation and provides examples of their use.
Transformation methods
The following table lists the transformations that Sensitive Data Protection provides to de-identify sensitive data:
| Transformation | Object | Description | Can Reverse1 | Referential Integrity2 | Input Type |
|---|---|---|---|---|---|
| Redaction | RedactConfig |
Redacts a value by removing it. | Any | ||
| Replacement | ReplaceValueConfig |
Replaces each input value with a given value. | Any | ||
| Replace with dictionary | ReplaceDictionaryConfig
|
Replaces an input value with a value that is randomly selected from a word list. | Any | ||
| Replace with infoType | ReplaceWithInfoTypeConfig |
Replaces an input value with the name of its infoType. | Any | ||
| Mask with character | CharacterMaskConfig |
Masks a string either fully or partially by replacing a given number of characters with a specified fixed character. | Any | ||
| Pseudonymization by replacing input value with cryptographic hash | CryptoHashConfig |
Replaces input values with a 32-byte hexadecimal string generated using a given data encryption key. See pseudonymization conceptual documentation to learn more. | ✔ | Strings or integers | |
| Pseudonymization by replacing with cryptographic format preserving token | CryptoReplaceFfxFpeConfig |
Replaces an input value with a token, or surrogate value, of the
same length using format-preserving encryption (FPE) with the FFX mode
of operation. This allows the output to be used in systems that have
format validation on length. This is useful for legacy systems where
string length must be maintained.
Important: For input that varies in length or has
length greater than 32 bytes, use CryptoDeterministicConfig.
To remain secure the following limits are recommended by the
National Institute of Standards and Technology:
|
✔ | ✔ | Strings or integers with a limited number of characters and of uniform length. The alphabet must be made up of at least 2 characters and contain no more than 95. |
| Pseudonymization by replacing with cryptographic token | CryptoDeterministicConfig |
Replaces an input value with a token, or surrogate value, of the same length using AES in Synthetic Initialization Vector mode (AES-SIV). This transformation method, unlike format-preserving tokenization, has no limitation on supported string character sets, generates identical tokens for each instance of an identical input value, and uses surrogates to enable re-identification given the original encryption key. | ✔ | ✔ | Any |
| Bucket values based on fixed size ranges | FixedSizeBucketingConfig |
Masks input values by replacing them with buckets, or ranges within which the input value falls. | Any | ||
| Bucket values based on custom size ranges | BucketingConfig |
Buckets input values based on user-configurable ranges and replacement values. | Any | ||
| Date Shifting | DateShiftConfig |
Shifts dates by a random number of days, with the option to be consistent for the same context. | ✔ Preserves sequence and duration |
Dates/Times | |
| Extract time data | TimePartConfig |
Extracts or preserves a portion of Date,
Timestamp, and TimeOfDay values. |
Dates/Times |
Footnotes
content.reidentify method.
Redaction
If you want to simply remove sensitive data from your input content,
Sensitive Data Protection supports a redaction transformation
(RedactConfig in the
DLP API).
For example, suppose you want to perform a simple redaction of all
EMAIL_ADDRESS infoTypes, and the following string is sent to
Sensitive Data Protection:
My name is Alicia Abernathy, and my email address is aabernathy@example.com.
The returned string will be the following:
My name is Alicia Abernathy, and my email address is .
The following JSON example and code in several languages shows how to form the API request and what the DLP API returns.
C#
To learn how to install and use the client library for Sensitive Data Protection, see Sensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.
Go
To learn how to install and use the client library for Sensitive Data Protection, see Sensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.