Azure Form Recognizer is a cognitive service provided by Microsoft Azure. It uses advanced machine learning technology to extract valuable information from different types of forms, such as invoices, receipts, and business cards. The service can identify and extract key-value pairs and tables from documents. It can process printed and handwritten text in various languages. This robust service is particularly useful in automating business processes, reducing manual data entry, and improving data accuracy.
Building a Custom Model
To begin with, it’s critical to understand that Azure Form Recognizer provides two types of models: prebuilt models and custom models. Prebuilt models identify and extract common fields from specific types of documents, like receipts or business cards. While these models are incredibly handy, they might not always cater to your organization’s specific needs. This is when building a custom model comes into play.
A custom model is trained with your own data, meaning it can learn to recognize the layout and structure of your documents across various types, like invoices or patient forms.
Training a custom model comprises two steps:
- Training: Here, you upload at least five different forms with the same layout to a blob storage. Azure Form Recognizer uses this data to understand the form’s layout and the location of important fields that need to be extracted.
- Analysis: After training the model, it’s used to analyze other forms with the same layout. The model takes the unseen form as an input and outputs the key and value pairs it has learnt to extract.
Here is sample code on how to create a custom model using Azure SDK:
from azure.ai.formrecognizer import FormTrainingClient
endpoint = "your_endpoint"
key = "your_key"
form_recognizer_client = FormTrainingClient(endpoint=endpoint, credential=AzureKeyCredential(key))
training_data_url = "blob_containing_sample_forms_sas_url"
poller = form_recognizer_client.begin_training(
training_data_url, use_training_labels=False)
model = poller.result()
print("Model ID: {}".format(model.model_id))
print("Status: {}".format(model.status))
print("Training started on: {}".format(model.training_started_on))
print("Training completed on: {}".format(model.training_completed_on))
In the above code, replace your_endpoint, your_key, and blob_containing_sample_forms_sas_url with your actual values.
Optimizing a Custom Model
Optimizing your custom model mostly involves improving the dataset you use to train the model. Here are a few strategies to consider when optimizing your custom model:
- Increase the Dataset: The more samples a model is trained on, the better it is capable of handling variations and unexpected data conditions. However, keep in mind, the samples should be representative of the forms you’re looking to analyze.
- Curate the Dataset: Curate your dataset to include different variations of form. The variations can be in terms of form layout, handwriting styles (for handwritten forms), and print quality. This will ensure that the model is more resilient to real-world variations.
- Use Labels: When training, you can opt to use labeled data. This involves associating specific fields in your input forms with labels. Using this labeled data can significantly improve the accuracy of your model.
- Iterative improvement: Start with a baseline model and then iteratively refine your model, based on the extracted results. Evaluate the outputs, see what it gets wrong, refine your data, and retrain, until you get satisfactory results.
Remember, building and optimizing a custom model for Azure Form Recognizer is an iterative process. Keep evaluating and refining your model to improve its performance gradually. By incorporating the Azure Form Recognizer into your tool chain, you can improve the efficiency and accuracy of your data input workflow, ultimately saving time and preventing errors.
Practice Test
True or False: Azure Form Recognizer is a cloud service that uses machine-learning to recognize and automate the extraction of data from common forms.
- Answer: True
Explanation: Form Recognizer precisely reads and extracts key-value pairs and table data from a variety of business documents like receipts, invoices, and contracts.
What is the process for training a custom model with Form Recognizer?
- A. Use labeled data
- B. Set the model id
- C. Test the model
- D. All of the above
Answer: D. All of the above
Explanation: When training a custom model with Form Recognizer, we first use labeled data. Then we train with that data and get a model id. Finally, we test this model for our requirements.
True or False: Azure Form Recognizer does not support extracting data from PDFs.
- Answer: False
Explanation: Form Recognizer supports extracting structured data from PDF files, along with TIFF and JPG images.
True or False: While building a custom model, you can either use form fields manually labeled by a human, only use unlabelled data, or use both.
- Answer: True
Explanation: Azure Form Recognizer allows for both options, which in turn allows enhancing the model’s accuracy.
Which of the following is not a step in optimizing a model for Azure Form Recognizer?
- A. Increasing training data
- B. Leveraging data variety
- C. Providing comprehensive labels
- D. Decreasing the test data
Answer: D. Decreasing the test data
Explanation: Test data helps validate the performance of the model. Reducing it will not contribute to optimization.
True or False: You can build and train models using the REST API and the SDKs provided by Azure Form Recognizer.
- Answer: True
Explanation: These are powerful tools provided by the service for building and training custom models.
True or False: The ‘Train Custom Model’ process of Azure Form Recognizer supports AutoML.
- Answer: False
Explanation: Currently, the ‘Train Custom Model’ process does not support AutoML.
Multiple select: Which of the following can be done to enhance the performance of Azure Form Recognizer models?
- A. Increase data variety
- B. Include erroneous labels
- C. Use large amount of training data
- D. Ensure data quality
Answer: A. Increase data variety, C. Use large amount of training data, D. Ensure data quality
Explanation: All these methods can enhance the model performance. Erroneous labels will rather decrease the performance.
In Azure Form Recognizer, which of the following can be used to export data?
- A. JSON
- B. PDF
- C. JPEG
- D. PNG
Answer: A. JSON
Explanation: Azure Form Recognizer supports data export in JSON format.
True or False: Models trained with Azure Form Recognizer are not reusable.
- Answer: False
Explanation: Models learned with Azure Form Recognizer are reusable and can be applied to multiple documents.
Interview Questions
What is Azure Form Recognizer?
Azure Form Recognizer is a cognitive service by Microsoft Azure that uses machine learning technology to identify and extract key-value pairs and tables from your documents.
Which types of models can you create with Azure Form Recognizer?
With Azure Form Recognizer you can create prebuilt models, custom models and compose models.
What is the purpose of training a custom model in Azure Form Recognizer?
The custom model is trained to automatically recognize and extract information from your documents, effectively tailoring its recognition and extraction capabilities to your specific needs.
How can one optimize a custom model in Azure Form Recognizer?
Optimization can be achieved by using a variety of machine learning techniques, such as incremental learning, where the model is continuously updated as it processes more data, and using larger, higher-quality training datasets which increase the model’s accuracy.
What types of documents does Azure Form Recognizer support?
Azure Form Recognizer supports several document formats including PDF, JPEG, PNG, and TIFF.
How does Azure Form Recognizer handle multiple languages?
Azure Form Recognizer supports English text out-of-the-box and can be trained to recognize other languages.
How can I make the custom model more accurate?
You can make a custom model more accurate by providing more labeled training data. You can also use manual review for recognition results and add these manually corrected forms to your training data.
What is the significance of labels in Azure Form Recognizer?
Labels are the truths used to train your model. The better your model understands the labels, the better your model will recognize and extract data from new unseen documents.
How to handle errors during training a custom model?
Azure provides detailed reporting of any errors occurred during the training process. Issues related to the quality of the data, the format of the documents or non-uniformity among the documents can often cause problems.
What Azure resources can you use to streamline manual review of documents processed by Azure Form Recognizer?
Azure Machine Learning’s Data Labeling feature allows you to manually review documents and add these labeled documents to your training data.
How can we access Form Recognizer service API?
Form Recognizer’s service API can be accessed via RESTful endpoint URLs provided on the Microsoft Azure platform.
What is the role of Azure Storage in using Azure Form Recognizer?
Azure Storage is used to store the input documents and labeled data that are used for training the Form Recognizer model.
Can text printed at any angle in the document/images be identified and extracted using Azure Form Recognizer?
Yes, Azure Form Recognizer supports the extraction of text printed at different angles.
How to improve performance of Form Recognizer when dealing with large amounts of data?
You can partition the data and process it concurrently in Azure to improve the performance when dealing with large amounts of data.
What is the process of deploying Azure Form Recognizer?
Azure Form Recognizer is a cloud-based service so there is no physical deployment necessary. You just need to create an instance of the service in your Azure account and train it to your needs.
extutorials.com