Author : HASSAN MD TAREQ | Updated : 2020/11/11
What is batch service?
- Azure Batch Service is a cloud based job scheduling and compute management platform that enables running large-scale parallel and high performance computing applications
- Developers can use Batch as a platform service to build SaaS applications or client apps where large-scale execution is require
- Batch Service is one of the High-Performance Computing options in Azure
- There is no additional charge for using Batch. You only pay for the underlying resources consumed, such as the virtual machines, storage, and networking
- HPC (high performace computing)
- Large-scale parallel workloads i.e.
- Data ingestion, processing, and ETL operations
- Genetic sequence analysis
- Image analysis and processing
- Media transcoding
- VFX and 3D image rendering
- Financial risk modeling
- Tightly coupled workloads i.e.
- Finite element analysis
- Fluid dynamics
- Multi-node AI training
- Large-scale rendering workloads with rendering tools i.e. Autodesk Maya, 3ds Max, Arnold, V-Ray etc.
- Node: In an Azure Batch workflow, a compute node (or node) is a virtual machine that processes a portion of your application’s workload
- Pool: A pool is a collection of nodes for applications to runs on (a set of virtual machines)
- Jobs: Tasks/workloads that need to be processed by Nodes/VMs
- Job schedules: Schedule is needed in-order to run the jobs in the pool
- Storage account: for storing input and output data
Steps in a common Batch workflow
[Image courtesy: Microsoft docs]
Batch job using PowerShell
Login-AzAccount # Login-AzureRmAccount Get-AzSubscription # Get-AzureRmSubscription Set-AzContext -SubscriptionName "IaaS for Devs" # Set-AzureRmContext $location = "westus" $rgName = "batch-ps-rg" $acctName = "storageacctps99999" $batchAcctName = "batchacctps201999" New-AzResourceGroup -Name $rgName -Location $location $storageAcct = New-AzStorageAccount -ResourceGroupName $rgName ` -Name $acctName ` -Location $location ` -SkuName Standard_LRS ` -Kind StorageV2 New-AzBatchAccount -ResourceGroupName $rgName ` -Name $batchAcctName ` -Location $location ` -AutoStorageAccountId $storageAcct.Id
Batch job using Azure CLI
az login az account list --output tsv az account set --subscription "IaaS for Devs" az group create --name batch-cli-rg --location westus az storage account create \ --resource-group batch-cli-rg \ --name storageacctcli12342 \ --location westus2 \ --sku Standard_LRS az batch account create \ --name batchacctcli12342 \ --storage-account storageacctcli12342 \ --resource-group batch-cli-rg \ --location westus2
Batch job using .Net