What is batch service?

• Azure Batch Service is a cloud based job scheduling and compute management platform that enables running large-scale parallel and high performance computing applications
• Developers can use Batch as a platform service to build SaaS applications or client apps where large-scale execution is require
• Batch Service is one of the High-Performance Computing options in Azure
• There is no additional charge for using Batch. You only pay for the underlying resources consumed, such as the virtual machines, storage, and networking

Usage

• HPC (high performace computing)
• Large-scale parallel workloads i.e.
  • Data ingestion, processing, and ETL operations
  • Genetic sequence analysis
  • Image analysis and processing
  • Media transcoding
  • VFX and 3D image rendering
  • Financial risk modeling
  • etc.
• Tightly coupled workloads i.e.
  • Finite element analysis
  • Fluid dynamics
  • Multi-node AI training
• Large-scale rendering workloads with rendering tools i.e. Autodesk Maya, 3ds Max, Arnold, V-Ray etc.

Concepts

• Node: In an Azure Batch workflow, a compute node (or node) is a virtual machine that processes a portion of your application’s workload
• Pool: A pool is a collection of nodes for applications to runs on (a set of virtual machines)
• Jobs: Tasks/workloads that need to be processed by Nodes/VMs
• Job schedules: Schedule is needed in-order to run the jobs in the pool
• Storage account: for storing input and output data

Steps in a common Batch workflow

[Image courtesy: Microsoft docs]

Batch job using PowerShell

Login-AzAccount       # Login-AzureRmAccount 

Get-AzSubscription    # Get-AzureRmSubscription

Set-AzContext -SubscriptionName "IaaS for Devs" # Set-AzureRmContext

$location = "westus"
$rgName = "batch-ps-rg"
$acctName = "storageacctps99999"
$batchAcctName = "batchacctps201999"

New-AzResourceGroup -Name $rgName -Location $location 

$storageAcct = New-AzStorageAccount -ResourceGroupName $rgName `
            -Name $acctName `
            -Location $location `
            -SkuName Standard_LRS `
            -Kind StorageV2

New-AzBatchAccount -ResourceGroupName $rgName `
            -Name $batchAcctName `
            -Location $location `
            -AutoStorageAccountId $storageAcct.Id

Batch job using Azure CLI

az login

az account list --output tsv

az account set --subscription "IaaS for Devs"

az group create --name batch-cli-rg --location westus

az storage account create \
    --resource-group batch-cli-rg \
    --name storageacctcli12342 \
    --location westus2 \
    --sku Standard_LRS

az batch account create \
    --name batchacctcli12342 \
    --storage-account storageacctcli12342 \
    --resource-group batch-cli-rg \
    --location westus2

Batch job using .Net

https://docs.microsoft.com/en-us/azure/batch/quick-run-dotnet