Terraform apply is stuck because of a state lock. Another engineer's apply crashed mid-execution. Fix it.

Step 1: Verify the Lock is Actually Orphaned

Before force-unlocking, confirm no apply is running:

# Check if the lock holder's process might still be running
# Contact the engineer if possible - their apply might be in progress!

# Check CloudWatch or your monitoring for recent Terraform activity
aws cloudwatch get-metric-statistics \
  --namespace AWS/DynamoDB \
  --metric-name ConsumedWriteCapacityUnits \
  --dimensions Name=TableName,Value=terraform-locks \
  --start-time 2024-01-15T10:00:00Z \
  --end-time 2024-01-15T11:00:00Z \
  --period 300 \
  --statistics Sum

# Check DynamoDB directly for lock details
aws dynamodb get-item \
  --table-name terraform-locks \
  --key '{"LockID": {"S": "s3://company-terraform-state/prod/vpc/terraform.tfstate-md5"}}'

Step 2: Force Unlock the State

# Use the Lock ID from the error message
terraform force-unlock a1b2c3d4-5678-90ab-cdef-1234567890ab

# You'll see a confirmation prompt:
Do you really want to force-unlock?
  Terraform will remove the lock on the remote state.
  This will allow local Terraform commands to modify this state, even though it
  may still be in use. Only 'yes' will be accepted to confirm.

  Enter a value: yes

# Output on success:
Terraform state has been successfully unlocked!

Step 3: Verify State Integrity

Critical: After force-unlock, immediately check for issues:

# Run plan to see current state vs reality
terraform plan

# If plan shows unexpected changes, the previous apply may have partially completed
# Look for resources that were being created/modified

# Check state list to see all managed resources
terraform state list

# For specific resources that might be affected:
terraform state show aws_vpc.main

Step 4: Handle Partial Apply Scenarios

If the previous apply partially completed:

# Option 1: If resources were partially created, import them
terraform import aws_instance.web i-1234567890abcdef0

# Option 2: If resources are in bad state, taint and recreate
terraform taint aws_instance.web
terraform apply

# Option 3: Refresh state to match reality
terraform refresh  # Deprecated in 1.5+, use:
terraform apply -refresh-only

Step 5: Implement Safeguards

Backend configuration with proper timeouts:

# backend.tf
terraform {
  backend "s3" {
    bucket         = "company-terraform-state"
    key            = "prod/vpc/terraform.tfstate"
    region         = "us-east-1"
    encrypt        = true
    dynamodb_table = "terraform-locks"

    # Reduce lock timeout for faster failure detection
    # Default is 0 (wait forever)
  }
}

Use Terraform Cloud/Enterprise for better lock management:

terraform {
  cloud {
    organization = "company"
    workspaces {
      name = "prod-vpc"
    }
  }
}

CI/CD pipeline with automatic lock release:

# .github/workflows/terraform.yml
jobs:
  terraform:
    runs-on: ubuntu-latest
    timeout-minutes: 30  # Pipeline timeout prevents infinite locks
    steps:
      - uses: actions/checkout@v4

      - name: Terraform Apply
        run: terraform apply -auto-approve
        timeout-minutes: 20  # Step-level timeout

      # Cleanup step runs even if apply fails
      - name: Release Lock on Failure
        if: failure()
        run: |
          # Log for audit trail
          echo "Apply failed, checking for orphaned lock..."
          # Don't auto-unlock - notify team instead
          # terraform force-unlock would be dangerous here