When working with Azure Machine Learning, one crucial step in your project pipeline is creating “compute targets” to run your experiments and training data. A compute target can be defined as the computing resource or environment where you run your training scripts or host your service deployment.
By setting and managing compute targets, you can structure your workflows and optimize your resource use across your machine learning project.
Types of Compute Targets in Azure Machine Learning
There are four types of compute targets in Azure Machine Learning:
- Local Compute: This uses the resources of the computer where you’re running the Azure Machine Learning code, be it your personal computer or a virtual machine.
- Azure Machine Learning Compute: This offers a cloud-based platform more suited for larger workloads where you need heavy computation.
- Azure Databricks: This provides an Apache Spark environment for big data processing and machine learning.
- Data Lake Analytics: This is a cloud-based service for big data analytics that simplifies the management, integration, and analysis of large and complex data sets.
You choose the compute target based on your requirements. For smaller datasets or quick tests, local compute might be adequate. For large-scale operations, you need to go for Azure Machine Learning Compute, Azure Databricks, or Data Lake Analytics.
Creating an Azure Machine Learning Compute in Python
For instance, to create an Azure Machine Learning Compute as your Compute target in Python, you use the following code.
from azureml.core import Workspace
from azureml.core.compute import ComputeTarget, AmlCompute
from azureml.core.compute_target import ComputeTargetException
#Specify the configuration for the new cluster
compute_config = AmlCompute.provisioning_configuration(vm_size=’STANDARD_D1_V2′,
min_nodes=0,
max_nodes=4)
#Create the cluster with the specified name and configuration
compute_target = ComputeTarget.create(ws, ‘my-compute-target’, compute_config)
#Wait for the cluster to be provisioned
compute_target.wait_for_completion(show_output=True)
In this example, we first define our requirements (the VM size and number of nodes) and then create the compute target with our desired name (‘my-compute-target’) and the predefined configuration. The `wait_for_completion(show_output=True)` line waits for the compute target to be completely provisioned before moving onto the rest of the code.
By defining a cloud-based compute target, you can take advantage of Azure’s scalability to run your experiments or train your models on demand, scaling up to handle larger workloads as needed.
Choosing Your Compute Target
Understand that the choice of your compute target does depend largely on your specific use case and application requirement. Examples add to understanding, but it is recommended that you explore Azure’s wide selection of compute target options based on your data volume, computational need, and other factors.
Conclusion
In conclusion, creating compute targets greatly simplifies the machine learning pipeline on Azure by managing computational resources efficiently. Whether it’s to run experiments or train models, understanding compute targets can help you effectively design and implement your data science solution on Azure.
Practice Test
True or False: Compute targets in Azure Machine Learning are the computational resources to run your training script or host your service deployment.
- True
- False
Answer: True
Explanation: Compute targets are resources / environments where you run your training script or host your service deployment.
Which of the following can be used as a compute target in Azure Machine Learning?
- A. Training clusters
- B. Inference clusters
- C. Both
Answer: C. Both
Explanation: Both training and inference clusters can be used as compute targets in Azure Machine Learning.
True or False: Compute targets are automatically managed by Azure.
- True
- False
Answer: True
Explanation: Azure takes care of maintaining, securing, and ensuring the availability of compute targets.
Which of the following is NOT a type of compute target in Azure Machine Learning?
- A. Training Cluster
- B. Inference Cluster
- C. Storage Cluster
Answer: C. Storage Cluster
Explanation: There is no concept of Storage Cluster as a compute target in Azure Machine Learning.
When using Azure Machine Learning, compute targets must be explicitly specified in data science experiments.
- True
- False
Answer: True
Explanation: Yes, we need to specify compute targets in order to tell Azure where to run the experiment or model training.
Which compute target would you use for low-cost, shared CPU resources suitable for running experiments and training models in development and test scenarios?
- A. Azure Machine Learning Compute Instance
- B. Azure Machine Learning Compute Cluster
Answer: A. Azure Machine Learning Compute Instance
Explanation: Azure Machine Learning Compute Instances are used in the development phase where we need low-cost, shared CPU resources.
What is the main benefit of using Compute Clusters as a compute target in Azure Machine Learning?
- A. They use shared CPU resources
- B. They provide on-demand, scalable resources
Answer: B. They provide on-demand, scalable resources
Explanation: Compute Clusters provides on-demand resources that automatically scale to meet the needs of your jobs.
True or False: You can use your local computer as a compute target in Azure Machine Learning.
- True
- False
Answer: True
Explanation: Local computers can be used as a compute target, which is useful for development and debugging.
Azure Machine Learning compute clusters are created with a minimum __ node(s) and can scale up to a maximum __ node(s).
- A. 0, 100
- B. 1, 500
- C. 0, 500
Answer: C. 0, 500
Explanation: The minimum number of nodes that can be specified when creating a cluster is 0 and the maximum number of nodes can be up to
Compute targets can be shared across different workspaces. True or False?
- True
- False
Answer: False
Explanation: Compute targets cannot be shared across workspaces. They are workspace-specific.
Azure Machine Learning provides which feature to automatically scale down compute resources when they are not in use?
- A. Scaling intervals
- B. Idle timeout
- C. Auto-pause
Answer: B. Idle timeout
Explanation: Idle timeout feature provided by Azure Machine Learning helps to automatically scale down the compute resources when they are not in use.
Compute Instances in Azure Machine Learning can be paused when not in use to reduce cost. True or False?
- True
- False
Answer: True
Explanation: Compute Instances hepls to reduce cost by pausing it when not in use.
Which of the following cannot be used as a compute target in Azure Machine Learning?
- A. Azure Kubernetes Service
- B. Azure Databricks
- C. Azure Blob Storage
Answer: C. Azure Blob Storage
Explanation: Azure Blob Storage is a data storage service, not a compute target.
True or False: Compute targets need to be created for each experiment.
- True
- False
Answer: False
Explanation: Compute targets are reusable and can be used for multiple experiments.
Compute targets in Azure Machine Learning need manual updating and maintenance. True or False?
- True
- False
Answer: False
Explanation: Azure Machine Learning manages compute targets automatically. Users don’t need to perform maintenance tasks on their side.
Interview Questions
What is the primary purpose of compute targets in Azure Machine Learning?
Compute targets are used for data experimentation and model training. These can be local or in the Azure cloud and provide the necessary compute resources to run your training script or host your service deployment.
What are the types of compute targets you can use in Azure Machine Learning?
The types of compute targets are Azure Machine Learning Compute, Azure Databricks, Azure Kubernetes Service (AKS), Azure HDInsight, Azure Data Lake Analytics, and virtual machines.
What is Azure Machine Learning Compute?
Azure Machine Learning Compute is a managed service that enables the ability to train machine learning models on clusters of Azure virtual machines.
Why would you use Azure Kubernetes Service as a compute target in Azure Machine Learning?
Azure Kubernetes Service (AKS) is used as a compute target when you need high-scale production deployments. It provides advanced networking, balancing, and scaling options suitable for complex deployments.
How can you create a compute target for your experiment in Azure Machine Learning studio?
The compute target can be created in Azure Machine Learning studio under the “Compute” section. You can choose the type of compute target, specify its size, and set the minimum and maximum number of nodes.
Can you use your local computer as a compute target in Azure Machine Learning service?
Yes, you can use your local machine as a compute target. This is particularly useful for development and testing purposes where large-scale compute resources are not required.
What happens when you attempt to run an experiment on a compute target that does not exist?
If a compute target does not exist, an error will be returned when you try to submit the experiment.
How does Azure Machine Learning handle scaling regarding the compute targets?
Azure Machine Learning features automatic scaling for compute targets. This means that Azure will determine the necessary resources based on the workload and will scale up or down as needed.
How do you specify a compute target when running an experiment in Azure Machine Learning?
You specify a compute target using the ‘compute_target’ parameter when running an experiment.
Can you change the compute target after an Azure Machine Learning experiment has started running?
No, the compute target cannot be changed once an experiment has started. You would need to stop the experiment, change the compute target, and then rerun the experiment.
What is the advantage of using Azure Databricks as a compute target?
Azure Databricks is a fast, easy, and collaborative Apache Spark-based analytics service that can be used as a compute target for large-scale machine learning tasks.
What type of compute targets would you use for low-priority or intermittent tasks in Azure Machine Learning?
You would use Azure BatchAI for low-priority or intermittent tasks as this service handles the distribution of the computation across a cluster.
Is it possible to delete a compute target in Azure Machine Learning?
Yes, it is possible to delete a compute target if it is not running or being used.
What compute target should you choose if you need to perform real-time inferencing in Azure Machine Learning?
For real-time inferencing, you should use Azure Kubernetes Service (AKS) as your compute target.
How do you manage costs with compute targets in Azure Machine Learning?
Azure Machine Learning provides options to control costs, such as setting a maximum number of nodes and enabling auto-pause for compute clusters when they are not in use.
extutorials.com