Implement a CI/CD pipeline with Azure DevOps that deploys to AKS with blue-green releases.

Q: Implement a CI/CD pipeline with Azure DevOps that deploys to AKS with blue-green releases.

Learn the answer to "Implement a CI/CD pipeline with Azure DevOps that deploys to AKS with blue-green releases." with detailed explanations, code examples, and best practices on DeployU.

The Scenario

Your team needs a robust deployment pipeline:

Current State:
├── Manual deployments via Portal/CLI
├── No automated testing
├── Secrets hardcoded in config files
├── Same deployment process for all environments
├── No rollback capability
├── Deployment frequency: 1-2 per month (fear of breaking things)
└── Production incidents during 30% of deployments

Target: Multiple deployments per day with zero-downtime and automatic rollback.

The Challenge

Build a production-ready CI/CD pipeline in Azure DevOps with proper branching strategy, security scanning, multi-stage deployments, approval gates, and rollback capabilities.

Wrong Approach

A junior engineer might create a single pipeline that deploys directly to production, store secrets in pipeline variables, skip testing stages, use the same configuration for all environments, or ignore security scanning. These approaches cause production incidents, security vulnerabilities, and lack auditability.

Addresses symptoms, not root cause

Right Approach

A senior engineer implements multi-stage pipelines with environment-specific configurations, integrates security scanning (SAST/DAST/SCA), uses Azure Key Vault for secrets, implements proper approval gates, and designs for zero-downtime deployments with rollback capabilities.

Step 1: Pipeline Architecture

CI/CD Pipeline Architecture:
┌─────────────────────────────────────────────────────────────────────────┐
│                        CI Pipeline (Build)                               │
│  ┌─────────┐  ┌─────────┐  ┌─────────┐  ┌─────────┐  ┌─────────┐       │
│  │ Restore │→ │  Build  │→ │  Test   │→ │Security │→ │Artifact │       │
│  │         │  │         │  │(Unit/   │  │ Scan    │  │Publish  │       │
│  │         │  │         │  │ Int)    │  │(SAST)   │  │         │       │
│  └─────────┘  └─────────┘  └─────────┘  └─────────┘  └─────────┘       │
└────────────────────────────────────────────────────────────────────────┘
                                    │
                                    ▼
┌─────────────────────────────────────────────────────────────────────────┐
│                        CD Pipeline (Release)                             │
│                                                                          │
│  ┌─────────────────┐      ┌─────────────────┐      ┌─────────────────┐ │
│  │   Development   │  →   │     Staging     │  →   │   Production    │ │
│  │                 │      │                 │      │                 │ │
│  │ • Auto deploy   │      │ • Auto deploy   │      │ • Manual gate   │ │
│  │ • Smoke tests   │      │ • Integration   │      │ • Blue-green    │ │
│  │ • No approval   │      │ • DAST scan     │      │ • Canary option │ │
│  └─────────────────┘      │ • Load test     │      │ • Rollback      │ │
│                           │ • 1 approval    │      │ • 2 approvals   │ │
│                           └─────────────────┘      └─────────────────┘ │
└─────────────────────────────────────────────────────────────────────────┘

Step 2: Multi-Stage YAML Pipeline

# azure-pipelines.yml
trigger:
  branches:
    include:
      - main
      - release/*
  paths:
    exclude:
      - docs/*
      - '*.md'

pr:
  branches:
    include:
      - main

variables:
  - group: 'global-variables'  # Variable group linked to Key Vault
  - name: buildConfiguration
    value: 'Release'
  - name: dotnetVersion
    value: '8.0.x'

stages:
  # ==================== BUILD STAGE ====================
  - stage: Build
    displayName: 'Build and Test'
    jobs:
      - job: BuildJob
        displayName: 'Build, Test, and Scan'
        pool:
          vmImage: 'ubuntu-latest'
        steps:
          - task: UseDotNet@2
            displayName: 'Install .NET SDK'
            inputs:
              version: $(dotnetVersion)

          - task: DotNetCoreCLI@2
            displayName: 'Restore packages'
            inputs:
              command: 'restore'
              projects: '**/*.csproj'
              feedsToUse: 'select'

          - task: DotNetCoreCLI@2
            displayName: 'Build solution'
            inputs:
              command: 'build'
              projects: '**/*.csproj'
              arguments: '--configuration $(buildConfiguration) --no-restore'

          - task: DotNetCoreCLI@2
            displayName: 'Run unit tests'
            inputs:
              command: 'test'
              projects: '**/*Tests.csproj'
              arguments: '--configuration $(buildConfiguration) --collect:"XPlat Code Coverage" --results-directory $(Agent.TempDirectory)'
              publishTestResults: true

          - task: PublishCodeCoverageResults@1
            displayName: 'Publish code coverage'
            inputs:
              codeCoverageTool: 'Cobertura'
              summaryFileLocation: '$(Agent.TempDirectory)/**/coverage.cobertura.xml'

          # Security scanning
          - task: CredScan@3
            displayName: 'Credential Scan'
            inputs:
              toolMajorVersion: 'V2'

          - task: SdtReport@2
            displayName: 'Security Analysis Report'
            inputs:
              GdnExportAllTools: true

          # Build and push Docker image
          - task: Docker@2
            displayName: 'Build Docker image'
            inputs:
              containerRegistry: 'acr-connection'
              repository: 'myapp/api'
              command: 'build'
              Dockerfile: '**/Dockerfile'
              tags: |
                $(Build.BuildId)
                $(Build.SourceBranchName)
                latest

          - task: Docker@2
            displayName: 'Push to ACR'
            inputs:
              containerRegistry: 'acr-connection'
              repository: 'myapp/api'
              command: 'push'
              tags: |
                $(Build.BuildId)
                $(Build.SourceBranchName)

          # Publish artifacts
          - task: PublishBuildArtifacts@1
            displayName: 'Publish artifacts'
            inputs:
              pathToPublish: '$(Build.ArtifactStagingDirectory)'
              artifactName: 'drop'

  # ==================== DEV STAGE ====================
  - stage: DeployDev
    displayName: 'Deploy to Development'
    dependsOn: Build
    condition: succeeded()
    variables:
      - group: 'dev-variables'
    jobs:
      - deployment: DeployDev
        displayName: 'Deploy to Dev'
        environment: 'development'
        pool:
          vmImage: 'ubuntu-latest'
        strategy:
          runOnce:
            deploy:
              steps:
                - task: AzureWebAppContainer@1
                  displayName: 'Deploy to App Service'
                  inputs:
                    azureSubscription: 'azure-dev-connection'
                    appName: 'app-myapi-dev'
                    containers: 'myacr.azurecr.io/myapp/api:$(Build.BuildId)'

                - task: AzureAppServiceManage@0
                  displayName: 'Start slot swap warmup'
                  inputs:
                    azureSubscription: 'azure-dev-connection'
                    Action: 'Start Azure App Service'
                    WebAppName: 'app-myapi-dev'

      - job: SmokeTest
        displayName: 'Smoke Tests'
        dependsOn: DeployDev
        pool:
          vmImage: 'ubuntu-latest'
        steps:
          - script: |
              response=$(curl -s -o /dev/null -w "%{http_code}" https://app-myapi-dev.azurewebsites.net/health)
              if [ $response -ne 200 ]; then
                echo "Health check failed with status $response"
                exit 1
              fi
              echo "Health check passed"
            displayName: 'Health check'

  # ==================== STAGING STAGE ====================
  - stage: DeployStaging
    displayName: 'Deploy to Staging'
    dependsOn: DeployDev
    condition: succeeded()
    variables:
      - group: 'staging-variables'
    jobs:
      - deployment: DeployStaging
        displayName: 'Deploy to Staging'
        environment: 'staging'
        pool:
          vmImage: 'ubuntu-latest'
        strategy:
          runOnce:
            deploy:
              steps:
                - task: AzureWebAppContainer@1
                  displayName: 'Deploy to staging slot'
                  inputs:
                    azureSubscription: 'azure-staging-connection'
                    appName: 'app-myapi-staging'
                    deployToSlotOrASE: true
                    slotName: 'staging'
                    containers: 'myacr.azurecr.io/myapp/api:$(Build.BuildId)'

      - job: IntegrationTests
        displayName: 'Integration Tests'
        dependsOn: DeployStaging
        pool:
          vmImage: 'ubuntu-latest'
        steps:
          - task: DotNetCoreCLI@2
            displayName: 'Run integration tests'
            inputs:
              command: 'test'
              projects: '**/*IntegrationTests.csproj'
              arguments: '--configuration $(buildConfiguration)'
            env:
              API_BASE_URL: 'https://app-myapi-staging-staging.azurewebsites.net'

      - job: LoadTest
        displayName: 'Load Test'
        dependsOn: IntegrationTests
        pool:
          vmImage: 'ubuntu-latest'
        steps:
          - task: AzureLoadTest@1
            displayName: 'Run load test'
            inputs:
              azureSubscription: 'azure-staging-connection'
              loadTestConfigFile: 'tests/load/config.yaml'
              resourceGroup: 'rg-loadtest'
              loadTestResource: 'lt-myapi'
              env: |
                [
                  { "name": "webapp", "value": "app-myapi-staging-staging.azurewebsites.net" }
                ]

      - job: SecurityScan
        displayName: 'DAST Security Scan'
        dependsOn: DeployStaging
        pool:
          vmImage: 'ubuntu-latest'
        steps:
          - task: owabormuwa.owasp-zap-azure-devops.owasp-zap-scan.owaspzap@1
            displayName: 'OWASP ZAP Scan'
            inputs:
              scanType: 'targetedScan'
              zapApiUrl: 'https://app-myapi-staging-staging.azurewebsites.net'

  # ==================== PRODUCTION STAGE ====================
  - stage: DeployProduction
    displayName: 'Deploy to Production'
    dependsOn: DeployStaging
    condition: and(succeeded(), eq(variables['Build.SourceBranch'], 'refs/heads/main'))
    variables:
      - group: 'prod-variables'
    jobs:
      - deployment: DeployProd
        displayName: 'Deploy to Production'
        environment: 'production'  # Has approval gates configured
        pool:
          vmImage: 'ubuntu-latest'
        strategy:
          runOnce:
            deploy:
              steps:
                # Deploy to staging slot first (blue-green)
                - task: AzureWebAppContainer@1
                  displayName: 'Deploy to production staging slot'
                  inputs:
                    azureSubscription: 'azure-prod-connection'
                    appName: 'app-myapi-prod'
                    deployToSlotOrASE: true
                    slotName: 'staging'
                    containers: 'myacr.azurecr.io/myapp/api:$(Build.BuildId)'

                # Warm up the staging slot
                - task: AzureAppServiceManage@0
                  displayName: 'Warm up staging slot'
                  inputs:
                    azureSubscription: 'azure-prod-connection'
                    Action: 'Start Azure App Service'
                    WebAppName: 'app-myapi-prod'
                    SpecifySlotOrASE: true
                    ResourceGroupName: 'rg-myapi-prod'
                    Slot: 'staging'

                - script: |
                    echo "Warming up staging slot..."
                    for i in {1..5}; do
                      curl -s https://app-myapi-prod-staging.azurewebsites.net/health
                      sleep 2
                    done
                  displayName: 'Warmup requests'

                # Swap slots (zero-downtime deployment)
                - task: AzureAppServiceManage@0
                  displayName: 'Swap slots'
                  inputs:
                    azureSubscription: 'azure-prod-connection'
                    Action: 'Swap Slots'
                    WebAppName: 'app-myapi-prod'
                    ResourceGroupName: 'rg-myapi-prod'
                    SourceSlot: 'staging'
                    PreserveVnet: true

            on:
              failure:
                steps:
                  # Automatic rollback on failure
                  - task: AzureAppServiceManage@0
                    displayName: 'Rollback - Swap back'
                    inputs:
                      azureSubscription: 'azure-prod-connection'
                      Action: 'Swap Slots'
                      WebAppName: 'app-myapi-prod'
                      ResourceGroupName: 'rg-myapi-prod'
                      SourceSlot: 'staging'

      - job: PostDeployValidation
        displayName: 'Post-Deployment Validation'
        dependsOn: DeployProd
        pool:
          vmImage: 'ubuntu-latest'
        steps:
          - script: |
              # Validate production health
              for i in {1..10}; do
                response=$(curl -s -o /dev/null -w "%{http_code}" https://app-myapi-prod.azurewebsites.net/health)
                if [ $response -eq 200 ]; then
                  echo "Production health check passed"
                  exit 0
                fi
                echo "Attempt $i failed, retrying..."
                sleep 10
              done
              echo "Production health check failed"
              exit 1
            displayName: 'Production health check'

Step 3: Environment Configuration

# environments/production.yml
# Azure DevOps Environment with approvals and checks

# Configure in Azure DevOps UI:
# 1. Go to Pipelines > Environments > production
# 2. Add Approvals and Checks:
#    - Approvals: Require 2 approvers from release-approvers group
#    - Business Hours: Only deploy Mon-Thu, 9AM-4PM
#    - Branch Control: Only from main branch
#    - Required Template: Must use approved pipeline template

# Variable group linked to Azure Key Vault
# azure-keyvault-prod:
#   - ConnectionString (secret)
#   - ApiKey (secret)
#   - AppInsightsKey (secret)

Step 4: Secure Secrets with Key Vault

# azure-pipelines.yml - Key Vault integration
variables:
  - group: 'azure-keyvault-prod'  # Linked to Key Vault

# In variable group settings:
# 1. Link to Azure Key Vault
# 2. Select secrets to expose
# 3. Secrets automatically rotated

steps:
  - task: AzureKeyVault@2
    displayName: 'Get secrets from Key Vault'
    inputs:
      azureSubscription: 'azure-prod-connection'
      KeyVaultName: 'kv-myapi-prod'
      SecretsFilter: 'SqlConnectionString,ApiKey'
      RunAsPreJob: true

  - task: AzureCLI@2
    displayName: 'Deploy with secrets'
    inputs:
      azureSubscription: 'azure-prod-connection'
      scriptType: 'bash'
      scriptLocation: 'inlineScript'
      inlineScript: |
        az webapp config appsettings set \
          --name app-myapi-prod \
          --resource-group rg-myapi-prod \
          --settings "SqlConnectionString=$(SqlConnectionString)"

Step 5: Canary Deployment Strategy

# Canary deployment using Traffic Manager
- stage: DeployProductionCanary
  displayName: 'Canary Deployment'
  jobs:
    - deployment: CanaryDeploy
      environment: 'production-canary'
      strategy:
        canary:
          increments: [10, 50, 100]  # Traffic percentages
          preDeploy:
            steps:
              - script: echo "Preparing canary deployment"
          deploy:
            steps:
              - task: AzureWebAppContainer@1
                inputs:
                  appName: 'app-myapi-prod-canary'
                  containers: 'myacr.azurecr.io/myapp/api:$(Build.BuildId)'
          routeTraffic:
            steps:
              - task: AzureCLI@2
                displayName: 'Route traffic to canary'
                inputs:
                  azureSubscription: 'azure-prod-connection'
                  scriptType: 'bash'
                  inlineScript: |
                    az webapp traffic-routing set \
                      --distribution canary=$(strategy.increment) \
                      --name app-myapi-prod \
                      --resource-group rg-myapi-prod
          postRouteTraffic:
            steps:
              - script: |
                  # Monitor error rate for 5 minutes
                  sleep 300
                  # Query Application Insights for error rate
                  error_rate=$(az monitor app-insights query \
                    --app appi-myapi-prod \
                    --analytics-query "requests | where timestamp > ago(5m) | summarize failure_rate = 100.0 * countif(success == false) / count()" \
                    --query "tables[0].rows[0][0]" -o tsv)

                  if (( $(echo "$error_rate > 1" | bc -l) )); then
                    echo "Error rate too high: $error_rate%"
                    exit 1
                  fi
                displayName: 'Monitor canary health'
          on:
            failure:
              steps:
                - task: AzureCLI@2
                  displayName: 'Rollback canary'
                  inputs:
                    azureSubscription: 'azure-prod-connection'
                    scriptType: 'bash'
                    inlineScript: |
                      az webapp traffic-routing clear \
                        --name app-myapi-prod \
                        --resource-group rg-myapi-prod

Step 6: Pipeline Templates for Reusability

# templates/build-template.yml
parameters:
  - name: dotnetVersion
    default: '8.0.x'
  - name: projectPath
    default: '**/*.csproj'
  - name: runTests
    type: boolean
    default: true

jobs:
  - job: Build
    pool:
      vmImage: 'ubuntu-latest'
    steps:
      - task: UseDotNet@2
        inputs:
          version: ${{ parameters.dotnetVersion }}

      - task: DotNetCoreCLI@2
        displayName: 'Build'
        inputs:
          command: 'build'
          projects: ${{ parameters.projectPath }}

      - ${{ if eq(parameters.runTests, true) }}:
          - task: DotNetCoreCLI@2
            displayName: 'Test'
            inputs:
              command: 'test'
              projects: '**/*Tests.csproj'

# Usage in main pipeline:
stages:
  - stage: Build
    jobs:
      - template: templates/build-template.yml
        parameters:
          dotnetVersion: '8.0.x'
          runTests: true

Systematic, production-ready debugging

Pipeline Security Checklist

Check	Implementation	Purpose
Secrets	Azure Key Vault + variable groups	Never in code
Branch protection	Required reviewers + status checks	Prevent direct pushes
Environment approvals	Manual gates for prod	Human verification
Security scanning	SAST/DAST/SCA in pipeline	Find vulnerabilities
Audit trail	Pipeline logs + Git history	Compliance

Deployment Strategies

Strategy	Downtime	Rollback Speed	Complexity
Basic	Yes	Slow	Low
Blue-Green (Slots)	Zero	Instant	Medium
Canary	Zero	Fast	High
Rolling	Minimal	Medium	Medium

Practice Question

Why should you deploy to a staging slot first and then swap slots for production deployments?

Questions