We need to test across multiple OS, Node versions, and configurations. Implement efficient matrix builds.

Q: We need to test across multiple OS, Node versions, and configurations. Implement efficient matrix builds.

Learn the answer to "We need to test across multiple OS, Node versions, and configurations. Implement efficient matrix builds." with detailed explanations, code examples, and best practices on DeployU.

The Scenario

Your library needs to support multiple environments:

Support Matrix:
- Node.js: 16, 18, 20
- OS: Ubuntu, Windows, macOS
- Package managers: npm, yarn, pnpm
- Databases: PostgreSQL 13, 14, 15, MySQL 8
Total combinations: 3 × 3 × 3 × 4 = 108 configurations

Running all 108 combinations for every PR would take hours and waste CI minutes. You need smart testing.

The Challenge

Implement an efficient matrix strategy that provides comprehensive coverage without excessive resource consumption.

Wrong Approach

A junior engineer might test all combinations for every commit, use sequential execution, or randomly pick a subset without analysis. These approaches waste resources, slow down feedback, or miss real compatibility issues.

Addresses symptoms, not root cause

Right Approach

A senior engineer implements tiered testing: quick smoke tests on PR, expanded matrix on main branch, full matrix for releases. Uses dynamic matrices, fail-fast for quick feedback, and excludes impossible combinations.

Step 1: Basic Matrix Configuration

name: CI Matrix

on: [push, pull_request]

jobs:
  test:
    runs-on: ${{ matrix.os }}
    strategy:
      matrix:
        os: [ubuntu-latest, windows-latest, macos-latest]
        node: [16, 18, 20]

    steps:
      - uses: actions/checkout@v4

      - name: Setup Node.js ${{ matrix.node }}
        uses: actions/setup-node@v4
        with:
          node-version: ${{ matrix.node }}

      - run: npm ci
      - run: npm test

Step 2: Optimize with Excludes and Includes

name: Smart Matrix

on: [push, pull_request]

jobs:
  test:
    runs-on: ${{ matrix.os }}
    strategy:
      fail-fast: false  # Don't cancel all jobs if one fails
      matrix:
        os: [ubuntu-latest, windows-latest, macos-latest]
        node: [16, 18, 20]

        # Exclude specific combinations
        exclude:
          # Node 16 is EOL, skip expensive Windows/Mac tests
          - os: windows-latest
            node: 16
          - os: macos-latest
            node: 16

        # Add specific test configurations
        include:
          # Test with experimental features on latest
          - os: ubuntu-latest
            node: 21
            experimental: true
          # Test specific npm version
          - os: ubuntu-latest
            node: 18
            npm: 10

    # Continue on experimental failures
    continue-on-error: ${{ matrix.experimental || false }}

    steps:
      - uses: actions/checkout@v4

      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: ${{ matrix.node }}

      - name: Use specific npm version
        if: matrix.npm
        run: npm install -g npm@${{ matrix.npm }}

      - run: npm ci
      - run: npm test

Step 3: Dynamic Matrix from JSON

name: Dynamic Matrix

on:
  push:
    branches: [main]
  pull_request:

jobs:
  # Generate matrix dynamically
  setup:
    runs-on: ubuntu-latest
    outputs:
      matrix: ${{ steps.set-matrix.outputs.matrix }}
    steps:
      - uses: actions/checkout@v4

      - name: Determine test matrix
        id: set-matrix
        run: |
          if [[ "${{ github.event_name }}" == "pull_request" ]]; then
            # Minimal matrix for PRs
            MATRIX=$(cat << 'EOF'
          {
            "os": ["ubuntu-latest"],
            "node": ["18"],
            "include": []
          }
          EOF
          )
          else
            # Full matrix for main branch
            MATRIX=$(cat << 'EOF'
          {
            "os": ["ubuntu-latest", "windows-latest", "macos-latest"],
            "node": ["16", "18", "20"],
            "exclude": [
              {"os": "windows-latest", "node": "16"},
              {"os": "macos-latest", "node": "16"}
            ]
          }
          EOF
          )
          fi
          echo "matrix=$(echo $MATRIX | jq -c .)" >> $GITHUB_OUTPUT

  test:
    needs: setup
    runs-on: ${{ matrix.os }}
    strategy:
      fail-fast: false
      matrix: ${{ fromJSON(needs.setup.outputs.matrix) }}
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: ${{ matrix.node }}
      - run: npm ci && npm test

Step 4: Matrix with Service Containers

name: Database Matrix

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        database:
          - engine: postgres
            version: 15
            port: 5432
          - engine: postgres
            version: 14
            port: 5432
          - engine: mysql
            version: 8
            port: 3306

    services:
      db:
        image: ${{ matrix.database.engine }}:${{ matrix.database.version }}
        env:
          POSTGRES_PASSWORD: ${{ matrix.database.engine == 'postgres' && 'postgres' || '' }}
          MYSQL_ROOT_PASSWORD: ${{ matrix.database.engine == 'mysql' && 'mysql' || '' }}
          MYSQL_DATABASE: ${{ matrix.database.engine == 'mysql' && 'test' || '' }}
        ports:
          - ${{ matrix.database.port }}:${{ matrix.database.port }}
        options: >-
          --health-cmd "${{ matrix.database.engine == 'postgres' && 'pg_isready' || 'mysqladmin ping' }}"
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5

    steps:
      - uses: actions/checkout@v4

      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: 18

      - run: npm ci

      - name: Run tests
        env:
          DB_ENGINE: ${{ matrix.database.engine }}
          DB_PORT: ${{ matrix.database.port }}
          DB_HOST: localhost
        run: npm run test:integration

Step 5: Tiered Testing Strategy

name: Tiered Testing

on:
  push:
    branches: [main, develop]
  pull_request:
  release:
    types: [published]

jobs:
  # Tier 1: Quick smoke tests (every commit)
  smoke:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 18
          cache: 'npm'
      - run: npm ci
      - run: npm run lint
      - run: npm run test:smoke

  # Tier 2: Standard tests (PR and main)
  standard:
    needs: smoke
    runs-on: ${{ matrix.os }}
    strategy:
      matrix:
        os: [ubuntu-latest]
        node: [18, 20]
        include:
          - os: windows-latest
            node: 18
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: ${{ matrix.node }}
          cache: 'npm'
      - run: npm ci
      - run: npm test

  # Tier 3: Extended tests (main branch only)
  extended:
    needs: standard
    if: github.ref == 'refs/heads/main'
    runs-on: ${{ matrix.os }}
    strategy:
      fail-fast: false
      matrix:
        os: [ubuntu-latest, windows-latest, macos-latest]
        node: [16, 18, 20]
        exclude:
          - os: macos-latest
            node: 16
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: ${{ matrix.node }}
      - run: npm ci
      - run: npm test
      - run: npm run test:integration

  # Tier 4: Full matrix (releases only)
  full:
    needs: extended
    if: github.event_name == 'release'
    runs-on: ${{ matrix.os }}
    strategy:
      fail-fast: false
      matrix:
        os: [ubuntu-latest, windows-latest, macos-latest]
        node: [16, 18, 20]
        package-manager: [npm, yarn, pnpm]
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-node@v4
        with:
          node-version: ${{ matrix.node }}

      - name: Install with ${{ matrix.package-manager }}
        run: |
          case "${{ matrix.package-manager }}" in
            npm)  npm ci ;;
            yarn) yarn install --frozen-lockfile ;;
            pnpm) corepack enable && pnpm install --frozen-lockfile ;;
          esac

      - run: npm test

Step 6: Matrix for Multiple Packages (Monorepo)

name: Monorepo Matrix

on: [push, pull_request]

jobs:
  detect-changes:
    runs-on: ubuntu-latest
    outputs:
      packages: ${{ steps.changes.outputs.packages }}
    steps:
      - uses: actions/checkout@v4

      - name: Detect changed packages
        id: changes
        run: |
          # Get list of changed packages
          PACKAGES=$(git diff --name-only HEAD~1 | grep '^packages/' | cut -d'/' -f2 | sort -u | jq -R -s -c 'split("\n")[:-1]')

          if [ "$PACKAGES" == "[]" ]; then
            PACKAGES='["core"]'  # Default if no specific changes
          fi

          echo "packages=$PACKAGES" >> $GITHUB_OUTPUT

  test:
    needs: detect-changes
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        package: ${{ fromJSON(needs.detect-changes.outputs.packages) }}
        node: [18, 20]

    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-node@v4
        with:
          node-version: ${{ matrix.node }}

      - name: Install dependencies
        run: npm ci

      - name: Test ${{ matrix.package }}
        run: npm run test --workspace=packages/${{ matrix.package }}

Systematic, production-ready debugging

Matrix Optimization Summary

Strategy	When to Use	Benefit
`exclude`	Skip impossible/unnecessary combos	Reduce jobs
`include`	Add specific test configs	Targeted coverage
`fail-fast: false`	When you need all results	Complete picture
Dynamic matrix	Different needs per event	Context-aware testing
Tiered testing	Balance speed vs coverage	Fast feedback + thorough releases

Practice Question

In a GitHub Actions matrix, what is the difference between 'exclude' and 'fail-fast'?

Questions