Custom extractor mechanisms

Create a processor

1. In the Google Cloud console, in the Document AI section, go to the Workbench page.

   Workbench

2. For custom extractor, select Create processor.

   

3. In the Create processor menu, enter a name for your processor, such as my-custom-document-extractor.

   

4. Select the region closest to you.

5. Optional: Open Advanced options.

   • You have the option to let Google create a Cloud Storage bucket for you, or you can create your own. For this tutorial, select Google-managed storage.

   • You also have the option to use Google-managed or Customer-managed encryption keys (CMEK). For this tutorial, select Google-managed encryption key.

6. Select Create to create your processor.

Document-level prompts

A document-level prompt provides a way to describe the document overall to bring deep business knowledge to the model and improve extraction quality. It can contain some general field information, for example: All the address fields are located in the USA.

While prompts can provide information to the model, they shouldn't be used to check that rules or conditions are met. Example: all prices should sum up to the total price. The document-level prompt is added to the overall prompt and should be treated as document-level hints, rather than a full prompt.

Provide a document-level prompt

1. In the Google Cloud console, go to the Get started tab.

2. To input the document prompt click Edit near the document prompt input field.

   

3. In the side-panel, provide a prompt that describes the document as a whole.

   

Define processor fields

You are now on the Processor overview page of the processor you just created.

You can specify the fields you want the processor to extract and begin labeling documents.

1. Select the Get started tab. The fields menu appears.

2. Select Create new field.

3. Enter the name for the field. Select the Data type and the Occurrence. Give the label a descriptive, distinct Description. Property description lets you provide additional context, insights, and prior knowledge for each entity to improve extraction accuracy and performance.

1. Select Create. Refer to Define processor schema for detailed instructions on creating and editing a schema.

1. Create each of the following labels for the processor schema.

   Name	Data Type	Occurrence
   control_number	Number	Optional multiple
   employees_social_security_number	Number	Required multiple
   employer_identification_number	Number	Required multiple
   employers_name_address_and_zip_code	Address	Required multiple
   federal_income_tax_withheld	Money	Required multiple
   social_security_tax_withheld	Money	Required multiple
   social_security_wages	Money	Required multiple
   wages_tips_other_compensation	Money	Required multiple

   You can also create and use other types of labels in your processor schema, such as checkboxes and tabular entities. For example, the W-2 forms contain statutory employee, retirement plan, and third party sick pay check boxes that you could also add to the schema.

   

Upload a sample document

Test with a sample document.

1. Select Upload sample document.

2. In the sidebar, select Import documents from Cloud Storage.

3. For this example, enter this bucket name in Source path. This links directly to one document.

   cloud-samples-data/documentai/Custom/W2/PDF/W2_XL_input_clean_2950.pdf
   

4. Select Import.

You are redirected to the labeling console.

Label a document

The process of selecting text in a document and applying labels is known as annotation or labeling.

1. When you're at the labeling console, notice that many of the labels are already populated. This is because the default custom extractor model type is a foundation model, which can perform zero-shot prediction, that is, without training.

   

2. To use the suggested labels, hold the pointer over each label in the side panel, and select the check mark to confirm the label is correct. Don't edit the text, even if the OCR reads the text incorrectly.

3. In this example, the values at the bottom of the document were not identified automatically, so you need to label them manually.

4. Use the icons in the toolbar above the document to label. Use the bounding box tool by default, or the Select text tool for multi-line values, to select the content and apply the label.

5. After text is selected, then a drop-down menu appears with all defined fields (entities) for you to select one. In this example, the value of wages_tips_other_compensation is selected with the bounding box tool, and that label is applied.

   

6. Review the detected text values to ensure that they reflect the correct location of text for each field. The labeled W2 document should look like this when complete:

   

7. If needed, you can select Create new field to add a new field to the schema from this page.

8. Select Mark as labeled when you have finished annotating the document. You are redirected to the Get started tab.

Build processor version using foundation model

After labeling a single document, you can create a processor version using the pretrained foundation model to extract entities.

1. Select the Build tab.

   

2. Under Call foundation model, select Create new version.

3. Enter a name for your processor version, such as w2-foundation-model.

4. Select Create version. It takes a few minutes to create.