Concepts

• Trigger: A trigger tells the pipeline when to run
• Pipeline: A pipeline defines the continuous integration and deployment process
• Stage: A stage is a collection of related jobs
• Job: Unit of work that can be scheduled to run on pipeline agent, consists of a set of steps
• Step: Smallest building block of a pipeline, can be a task or script
  • Task: A task is a pre-packaged script that performs an action
  • Script: A script runs code as a step in your pipeline using command line (PowerShell or Bash)
• Run: A run represents one execution of a pipeline
• Artifact: An artifact is a collection of files or packages published by a pipeline run
• https://docs.microsoft.com/en-us/azure/devops/pipelines/get-started/key-pipelines-concepts?view=azure-devops

Hierarchy

Pipeline
→ Stages
→ → Jobs
→ → → Steps (Tasks, Scripts)

[Image courtesy: Microsoft doc.]

About YAML pipeline

• Pipeline
  • A pipeline is made up of one or more stages
  • A pipeline can deploy to one or more environments
• YAML pipeline
  • Pipeline-as-Code
  • Simply add “azure-pipelines.yml” file to the root of the repository
  • Azure DevOps provides default templates for popular project types and a YAML designer to simplify the process of defining build and release tasks
  • Azure Pipelines doesn’t support all YAML features. Unsupported features include anchors, complex keys, and sets
  • Azure Pipelines depends on seeing stage, job, task, or a task shortcut like script as the first key in a mapping
• Links:
  • https://docs.microsoft.com/en-us/azure/devops/pipelines/get-started/what-is-azure-pipelines
  • https://docs.microsoft.com/en-us/azure/devops/pipelines/get-started/key-pipelines-concepts?view=azure-devops#pipeline

Pipeline structure: https://docs.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=azure-devops&tabs=schema%2Cparameter-schema#pipeline-structure

Syntax

resources: # define reosurces (i.e. variable group, pipeline etc.) that can be used in this pipline as resource
  xxx:
  - x:
    y:
  variables:
  - group: "demo-vg"

pool:
  vmImage: 'ubuntu-latest'
  
trigger:
  - branchName

stages:

- stage: Build
  displayName: My Build Stage
  jobs:
  - job: BuildJob
    displayName: My Build Job
    steps:
    - task: xxx
    - script: xxx

	
- stage: Test
  displayName: My Test Stage
  jobs:
  - job: TestJobOne
    displayName: My Test Job 1
    steps:
    - script: xxx
    - task: xxx
	
  - job: TestJobTwo
    displayName: My Test Job 2
    steps:
    - script: xxx
    - task: xxx
	
- stage: Deploy
  displayName: My Deploy Stage
  jobs:
  - job: DeployJob
    displayName: My Deploy Job
    continueOnError: false
    steps:
    - script: xxx
    - task: xxx

Trigger

Details: YAML pipeline trigger

Stage

• A stage is a logical groupING of related jobs. It can be used to mark separation of concerns (for example, Build, QA, and production)
• Stages are the major divisions in a pipeline. Each stage contains one or more jobs
• A stage is a way of organizing jobs in a pipeline and each stage can have one or more jobs
• Example of stages:
  • Build app
  • Run tests
  • Deploy to preproduction
• The build stages create the build artifacts and publish them (some are pipeline artifacts, others are docker images) and the deployment stages use the artifacts to deploy/release
• When you define multiple stages in a pipeline, by default, they run one after the other. You can specify the conditions for when a stage runs
• By default, stages run sequentially. Each stage starts only after the preceding stage is complete unless otherwise specified via the dependsOn property

Example: https://docs.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=azure-devops&tabs=example%2Cparameter-schema#stage

Job

• Job is the smallest unit of work that can be scheduled to run. A stage contains one or more jobs
• A job represents an execution boundary of a set of steps
• A job is a series of steps that run sequentially as a unit
• Each job runs on one agent. A job can also be agentless.
• A set of tasks to be performed in sequence to acommplish a deployment
• Pipeline jobs runs on pipeline agent. All of the steps run together on the same agent
• Example of multiple job in a stage:
  • build two configurations - x86 and x64
  • one stage and two jobs. One job would be for x86 and the other job would be for x64

Example: https://docs.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=azure-devops&tabs=example%2Cparameter-schema#job

Step

• A step is the smallest building block of a pipeline
• A linear sequence of steps make up a job
• A step can either be a script or a task
  • Script: a script runs code as a step in pipeline using command line, PowerShell or Bash. a script is custom code that is specific to your pipeline
  • Task: A task is packaged script or procedure that has been abstracted with a set of inputs

Example: https://docs.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=azure-devops&tabs=example%2Cparameter-schema#steps

Task

• Tasks are the building blocks of a pipeline
• Tasks consists a job and a job runs in pipeline agent VM (or PC)

Example: https://docs.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=azure-devops&tabs=example%2Cparameter-schema#task

Script

• ‘script’ is a shortcut for the Azure DevOps ‘command-line task’
• Command-line task runs a script using cmd.exe on Windows and Bash on other platforms

Example: https://docs.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=azure-devops&tabs=example%2Cparameter-schema#script

Deployment

• For YAML pipelines, a deployment typically refers to a deployment job
• A deployment job is a collection of steps that are run sequentially against an environment
• Details: https://docs.microsoft.com/en-us/azure/devops/pipelines/process/deployment-jobs?view=azure-devops

Resource

• A resource is anything used by a pipeline that lives outside the pipeline
• Example of Resource: Service connection, Environment, Variable group etc.
• Details: https://docs.microsoft.com/en-us/azure/devops/pipelines/process/about-resources?view=azure-devops&tabs=yaml

Environment:

• An environments represents a group of resources that is deployment target and to which artifacts will be deployed
• Use Ennvironment (Kubernetes Resource) instead of Kubernetes Service Connection for Deployment Job
• Details: Azure DevOps Environment

CI and CD

• CI (Continuous Integration)
  • Build and tests
  • Automated tests to ensure quality
  • Produce artifacts (fed to release processes to drive frequent deployments)
• CD (Continuous Delivery)
  • Build, test, and deploy to one or more environments
  • Automated release of artifacts (produced by CI) to existing systems (new versions / fixes)
• Create CI pipeline
• Create CD pipeline

Links

• Key concepts: https://docs.microsoft.com/en-us/azure/devops/pipelines/get-started/key-pipelines-concepts?view=azure-devops
• YAML pipeline editor: https://docs.microsoft.com/en-us/azure/devops/pipelines/get-started/yaml-pipeline-editor?view=azure-devops
• https://docs.microsoft.com/en-us/azure/devops/pipelines/get-started/multi-stage-pipelines-experience
• Stage: https://docs.microsoft.com/en-us/azure/devops/pipelines/process/stages?view=azure-devops&tabs=yaml
• Job: https://docs.microsoft.com/en-us/azure/devops/pipelines/process/phases?view=azure-devops&tabs=yaml
• Task:
  • https://docs.microsoft.com/en-us/azure/devops/pipelines/process/tasks?view=azure-devops&tabs=yaml
  • Catalog of tasks : https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/?view=azure-devops
• Script: https://docs.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=azure-devops&tabs=example%2Cparameter-schema#script
• Define a pipelines resource: https://docs.microsoft.com/en-us/azure/devops/pipelines/process/resources?view=azure-devops&tabs=schema#define-a-pipelines-resource