Extract information using prebuilt models in Azure Form Recognizer

Extracting information from forms and documents is an essential part of many businesses and industries. This may involve understanding and interpreting data from invoices, receipts, and other forms of paperwork. By employing AI, we can automate this process and significantly speed up data extraction, reducing manual labor and error. One such tool that makes this possible is Azure’s prebuilt models in Form Recognizer. This service uses machine learning technology to identify and extract key-value pairs and tables from documents.

What are Prebuilt Models in Azure Form Recognizer?

Microsoft Azure Form Recognizer’s prebuilt models are trained machine learning models designed to extract relevant information from specific types of forms. These include receipt model (for extracting data from sales receipts), invoice model (for extracting data from invoices), and business card models (for extracting contact information from business cards).

Furthermore, Azure Form Recognizer has a prebuilt document model. This general-purpose model can extract key-value pairs, tables, selection marks, single and multiline text from forms and documents, even if their layouts differ from one page to another.

How to use Azure Form Recognizer Prebuilt Models

Using Azure Form Recognizer’s prebuilt models involves a straightforward process:

• Select the Prebuilt Model: Choose the prebuilt model that fits the type of form or document from which you want to extract information.
• Send Document: Sending the document via POST request to the model in Azure Form Recognizer.
• Receive Extracted Data: The service then returns a structured JSON response containing the extracted data.

The following sections will delve into how to use the various prebuilt models in Azure Form Recognizer.

Azure’s Receipt Model

The Receipt model, which comes under Azure’s Form Recognizer’s prebuilt models, has been trained specifically on sales receipts from the United States. It can examine and identify important fields such as Merchant Name, Transaction Date, Items, Total, and more.

Here’s an example of how to use the Receipt model via a POST request:

url = "/formrecognizer/v2.1/prebuilt/receipt/analyze"
data = open("", "rb").read()
response = requests.post(url, headers={"Ocp-Apim-Subscription-Key": , "Content-Type": "application/octet-stream"}, data=data)


Azure’s Invoice Model

The Invoice model, another prebuilt model under Azure Form Recognizer, is designed to extract data from invoices. It can identify fields like Vendor Name, Customer Name, Invoice Date, Invoice Total, and more.

Here’s how to use the Invoice model with a POST request:

url = "/formrecognizer/v2.1/prebuilt/invoice/analyze"
data = open("", "rb").read()
response = requests.post(url, headers={"Ocp-Apim-Subscription-Key": , "Content-Type": "application/octet-stream"}, data=data)


Azure’s Business Card Model

The Business Card model in Azure Form Recognizer is trained to extract contact information from business cards in English. This can be quite handy for potential leads, contacts, and related scenarios.

Here’s a sample of how to use the Business Card model with a POST request:

url = "/formrecognizer/v2.1/prebuilt/businessCard/analyze"
data = open("", "rb").read()
response = requests.post(url, headers={"Ocp-Apim-Subscription-Key": , "Content-Type": "application/octet-stream"}, data=data)


Azure’s Document Model

The Document model is a powerful, versatile model that can extract key-value pairs, tables, and other forms of information from documents, regardless of their structure or format.

Here’s an example of how to use the Document model with a POST request:

url = "/formrecognizer/v2.1/prebuilt/document/analyze"
data = open("", "rb").read()
response = requests.post(url, headers={"Ocp-Apim-Subscription-Key": , "Content-Type": "application/octet-stream"}, data=data)


This overview of Azure Form Recognizer’s prebuilt models offers just a glimpse of their usefulness in extracting valuable information from various shapes and forms of documents. By leveraging these prebuilt models, professionals preparing for the AI-102 Designing and Implementing an Azure AI Solution exam can gain practical insights into handling real-world scenarios.

Practice Test

True or False: Azure Form Recognizer can analyze business cards, receipts, and other documents.

• True
• False

Answer: True

Explanation: The Azure Form Recognizer allows for analysis and extraction of key information from various document types, including business cards, receipts, invoices, and more.

What are the types of prebuilt models available in Azure Form Recognizer?

• a) Invoice
• b) Receipt
• c) Business card
• d) Driving license

Answer: a) Invoice, b) Receipt, c) Business card

Explanation: Prebuilt models in Azure Form Recognizer currently support extraction of information from business cards, invoices, and receipts. There is no prebuilt model for driving licenses.

Can Azure Form Recognizer prebuilt models analyze images that aren’t in English?

• a) Yes, but only few languages support.
• b) No, only English is supported.
• c) Yes, all languages are supported.
• d) No information provided by Microsoft.

Answer: a) Yes, but only few languages support.

Explanation: The Azure Form Recognizer currently supports several languages, not just English. However, it does not support all languages.

True or False: The Azure Form Recognizer does not provide a prebuilt model for table detection and extraction.

• True
• False

Answer: False

Explanation: Azure Form Recognizer does provide a prebuilt model for table extraction as part of its layout API.

True or False: Azure Form Recognizer requires manual data labeling before training a model.

• True
• False

Answer: False

Explanation: With Azure Form Recognizer, you do not need to manually label your data before training a model, especially when using prebuilt models.

What capabilities does Azure Form Recognizer provide?

• a) Text recognition
• b) Form layout understanding
• c) Data extraction
• d) Sentiment Analysis

Answer: a) Text recognition b) Form layout understanding c) Data extraction

Explanation: Azure Form Recognizer provides features like text recognition, form layout understanding, and data extraction but it does not provide sentiment analysis.

True or False: Azure Form Recognizer can only analyze printed text and cannot analyze handwritten text.

• True
• False

Answer: False

Explanation: Azure Form Recognizer can also analyze and recognize handwritten text as well as printed text.

True or False: You can extend the capabilities of prebuilt models in Azure Form Recognizer using training.

• True
• False

Answer: True

Explanation: Yes, you can extend the capabilities of prebuilt models by training them with your own data to tailor them to your specific needs.

Which item among the following cannot be recognized by the Azure Form Recognizer prebuilt receipt model?

• a) Time
• b) Date
• c) Merchant Name
• d) Colors of items

Answer: d) Colors of items

Explanation: The Azure Form Recognizer can recognize time, date, and merchant name from receipts, but there is no current support for recognizing colors of items.

True or False: Using Azure Form Recognizer’s prebuilt models, you can extract key-value pairs and tables from documents.

• True
• False

Answer: True

Explanation: Yes, Azure Form Recognizer’s prebuilt models do provide the functionality to extract key-value pairs and tables from documents.

Interview Questions

What is Azure Form Recognizer?

Azure Form Recognizer is an AI-powered document extraction service that understands your forms, enabling you to extract text, key-value pairs, and tables from your documents. The service uses machine learning technology to identify and extract information.

What are prebuilt models in Azure Form Recognizer?

Prebuilt models in Azure Form Recognizer are machine learning models that have been trained on a variety of data and are ready to analyze your documents for specific types of information, such as invoices, receipts, or business cards.

How can you use a prebuilt model to extract information from a document in Azure Form Recognizer?

You can use a prebuilt model by sending an API request that includes the URL of your document. The service will process your document and return a JSON output containing the extracted data.

Can you use both prebuilt and custom models in the same project?

Yes, you can combine prebuilt and custom models to handle a wide variety of data types and formats in the same project.

What types of prebuilt models are currently available in Azure Form Recognizer?

Azure Form Recognizer currently offers prebuilt models for business cards, identity documents, invoices, receipts, as well as support for general prebuilt models.

What type of input does Azure Form Recognizer support?

Azure Form Recognizer supports inputs in TIFF, PDF, PNG, and JPEG file formats.

How to prepare the data before using the prebuilt models to extract information in Azure Form Recognizer?

The forms should be in high quality, text must be easily legible, and the document structure should not be broken.

Where does Azure Form Recognizer return the extracted information from a document?

Azure Form Recognizer returns the extracted information in a JSON format, which includes the text, bounding box coordinates, confidence scores, and other data.

Can Azure Form Recognizer work with documents in different languages?

Yes, Azure Form Recognizer can work with documents in multiple languages including English, Spanish, German, French, Italian, Dutch, and Portuguese.

What is the use of bounding box in Azure Form Recognizer?

The bounding box represents the region of the analyzed document where the extracted text and other data elements are found.

What is the use of confidence score in Azure Form Recognizer?

The confidence score indicates the confidence of the model in its prediction. The closer the score is to 1.0, the more confident the model is in the accuracy of the extracted data.

How can you improve your results with Azure Form Recognizer?

You can improve your results by using a custom model trained specifically on your own form types, ensuring high-quality input data, and combining the results from multiple models.

Is it possible to fine-tune prebuilt models in Azure Form Recognizer?

No, prebuilt models cannot be fine-tuned. If you need more specific extraction, you should consider using the training APIs to create a custom model.

Is it possible to extract tables using prebuilt models in Azure Form Recognizer?

Yes, you can extract tables and their associated data using prebuilt models in Azure Form Recognizer.

Are prebuilt models in Azure Form Recognizer trainable?

No, prebuilt models are not trainable. They are already trained on a variety of data types and are ready for immediate use.