One of the crucial tasks you should be comfortable with is configuring an environment for a job run. This involves setting up your Azure Machine Learning workspace, creating an experiment, and executing a Python script in a given environment. This article will break down these processes for you.
Creating an Azure Machine Learning Workspace
The workspace is a central hub where you can manage all your Azure Machine Learning resources. Creating your workspace involves these general steps:
- Sign in to your Azure portal.
- Select “Create a resource” and then “Machine Learning”.
- Fill in necessary information like workspace name, resource group, location, and storage account.
Creating an Experiment
An experiment is a named process that runs scripts to train one or multiple machine learning models. It contains a run history which records details such as run comparison, run details, metrics, and outputs.
Use the following code to create the experiment:
from azureml.core import Experiment
experiment = Experiment(workspace=ws, name='my-experiment')
Here, `ws` stands for the workspace you have created in the previous step and `’my-experiment’` is the name of your experiment.
Setup the Environment
Before running the experiment, you will need to configure a Python environment. This environment will include all the necessary packages that your model will need to train and score. Azure ML environments are run within Docker containers by default which helps in providing a highly reproducible way to run your experiment.
You could create and manage your environment with Python as follows:
from azureml.core import Environment
myenv = Environment(name="myenv")
This creates an environment object. You can also specify a series of packages as dependencies within an environment as follows:
myenv.python.conda_dependencies.add_pip_package("scikit-learn")
This adds the scikit-learn package to this environment.
Creating a Run Configuration
A RunConfiguration object is used to configure the environment for your job run. It defines the compute target, environment, and other settings for a run. Create a RunConfiguration and connect it to your environment as follows:
from azureml.core.runconfig import RunConfiguration
run_config = RunConfiguration()
run_config.environment = myenv
run_config.target = compute_target
Starting a Run
The “run” is the execution of a particular script with a given run configuration. The script can be anything ranging from data pre-processing to training a complex machine learning model.
To start a run, you must first submit your script along with the run configuration to the experiment as follows:
run = experiment.submit(config=run_config, script='train.py')
Here, ‘train.py’ is your python script for a training a ML model.
Checking the Run
Once a run is submitted, you can monitor it using Azure Portal, Azure ML Studio, or programmatically using Python SDK.
run.wait_for_completion(show_output=True)
Here, `run.wait_for_completion(show_output=True)` method can be used to wait for the run to complete and view any logs generated during the operation.
By understanding these concepts and being able to configure an environment for a job run, you’ll be in a good position to succeed in the DP-100 Designing and Implementing a Data Science Solution on Azure exam.
Practice Test
True/False: Azure Machine Learning service allows for the configuration of multiple environments to run training scripts.
- True
Answer: True
Explanation: Azure Machine Learning service does allow for the configuration of multiple environments for training scripts.
Which of the following are steps in configuring an environment for a job run on Azure?
- a) Creating an Azure resource group
- b) Creating a workspace
- c) Creating a computer cluster
- d) Creating a data store
Answer: a, b, c, and d
Explanation: All these steps are part of the process in configuring an environment for a job run on Azure.
True/False: In Azure, environments are defined by YAML files.
- True
Answer: True
Explanation: YAML (yet another markup language) is used to define environments in Azure data science applications.
Which of the following Azure services cannot be used to run a data science job?
- a) Logic Apps
- b) Azure Functions
- c) Azure Machine Learning Service
- d) Azure Batch
Answer: a) Logic Apps
Explanation: While Logic Apps can be used to automate tasks and workflows, they are not specifically designed to support data science jobs.
Which of the following can be configured for a job run on Azure?
- a) Cluster
- b) Workspace
- c) Datasets
- d) Datastore
- e) All of the above
Answer: e) All of the above
Explanation: Azure Machine Learning service allows the configuration of various elements like cluster, workspace, datasets, and datastore for job runs.
True/False: After creating an Azure workspace, you do not need to create a compute target for running your training script.
- False
Answer: False
Explanation: You certainly need a compute target where you will run your training script after creating a workspace.
An Azure environment file for job run contains which of the following?
- a) Python packages
- b) Environment variables
- c) Docker settings
- d) All of the above
Answer: d) All of the above
Explanation: An Azure environment file contains a collection of all these.
True/False: Azure Data Factory is a service designed to configure environments for running data science jobs.
- False
Answer: False
Explanation: Azure Data Factory is a cloud-based data integration service and is not specifically designed to configure environments for running data science jobs.
Which of the following is used to train models in Azure?
- a) Virtual machines
- b) Container instances
- c) AKS
- d) All of the above
Answer: d) All of the above
Explanation: All these Azure services can be used to train models.
True/False: Azure Machine Learning workbench allows to configure an environment for a job run.
- False
Answer: False
Explanation: Actually, Azure Machine Learning workbench has been retired and replaced with Azure Machine Learning studio.
Which of the following is not a type of compute target in Azure?
- a) Training cluster
- b) Inferencing cluster
- c) Pipe cluster
- d) Compute instance
Answer: c) Pipe cluster
Explanation: Azure does not provide the concept of a Pipe clusters. Training clusters, Inferencing clusters, and Compute instances are all types of compute targets.
Which of the following is required to configure an environment for a job run in Azure?
- a) Azure portal
- b) Azure account
- c) Azure subscription
- d) All of the above
Answer: d) All of the above
Explanation: You need an Azure account with an active subscription and you use the Azure portal to configure the environment.
True/False: Azure CLI can be used to configure environments for running data science jobs.
- True
Answer: True
Explanation: The Azure command line interface (CLI) can indeed be used to configure environments and run data science jobs.
Does Azure support running data science jobs in a hybrid environment?
- a) Yes
- b) No
Answer: a) Yes
Explanation: Azure does support hybrid environments that utilize both cloud and on-premises resources.
True/False: Once a run configuration is created in Azure, it cannot be modified.
- False
Answer: False
Explanation: Azure does allow the modification of run configurations after they have been created.
Interview Questions
What is the significance of the Azure Machine Learning workspace in setting up an environment for a job run?
The Azure Machine Learning workspace is a central location for managing all artifacts created as part of an ML experiment in Azure. It ties together data, code, models and the compute resources required to run jobs in Azure Machine Learning.
How would you provision cloud-based compute resources using Azure Machine Learning SDK?
You can provision Azure Machine Learning compute instances or Azure Machine Learning compute clusters using the AzureML SDK. The ComputeTarget class of the SDK is used to interface with existing compute targets or to provision new ones.
What are the primary components of a Conda YAML file that’s used to configure the Python environment for a job run in Azure Machine Learning?
The primary components of a Conda YAML file are:
– Channels: Specifies where Conda should look for packages.
– Dependencies: Lists out all packages required for your environment.
How can you use Docker containers in the configuration of an Azure Machine Learning environment?
Docker containers can be utilized in Azure ML by specifying the use of Docker in the environment configuration. Docker provides a consistent, reproducible environment that’s isolated from the host OS, making it ideal for computational tasks in Azure ML.
How are environment configurations defined in Azure Machine Learning?
Environment configurations can be defined in Azure Machine Learning using the Environment class of the Azure ML SDK. It offers you full control over the specifics of your training environment, including the Python or R version to use, and any necessary packages or modules.
What is Azure Machine Learning Compute Instance?
Azure Machine Learning Compute Instance is a managed cloud-based workstation optimized for machine learning development. It provides developers with a consistent and secure environment to run ML jobs.
What do the pip and conda sections in the dependencies attribute of an Environment object represent?
The pip and conda sections list the packages that should be installed within the environment from the Python Package Index and the Conda package manager respectively.
How can you share an environment across workspaces in Azure Machine Learning?
Sharing an environment across workspaces can be done by registering the environment using the register() function. Other workspaces can then access the environment configuration, ensuring collaboration and reproducibility.
What is the role of Azure Machine Learning Studio in configuring an environment for a job run?
Azure Machine Learning Studio provides a visual interface to create, manage, and view all components associated with Azure Machine Learning, including environments. It enables users to configure a job run without dealing with the underlying code.
How can you clone an existing environment in Azure Machine Learning?
You can clone an existing environment in Azure Machine Learning using the clone() method on an Environment object. Cloning environments allows you to create a new environment that’s an exact copy of the existing one.
How can you control the Docker base image used by Azure Machine Learning for the execution environment?
You can specify the Docker image to be used for the execution environment within the Environment object’s docker subsection. This way, you can ensure that the Docker base image includes any OS-level dependencies required by your job.
How would you reset a Python environment in Azure Machine Learning?
To reset a Python environment in Azure Machine Learning, you could delete and recreate the environment, or create a new version of the environment with the updated specifications.
What purpose does the inferencing functionality in Azure Machine Learning serve?
Inferencing in Azure Machine Learning allows you to make predictions on your model. It involves configuring an inferencing environment that meets the resource requirements of the model you plan on serving.
What are the benefits of using scripts for environment creation in Azure Machine Learning?
Scripts for environment creation in Azure Machine Learning allow for effortless reproducibility. It ensures that the environment can be easily recreated, either in same workspace or shared across others, without any inconsistencies.
How can you specify GPU requirements in an Azure Machine Learning environment?
You can specify GPU requirements within the Azure Machine Learning environment’s docker section. Set the enabled field to True under the base_image field, which configures the environment to use a Docker image that includes GPU-specific dependencies.
extutorials.com