Your Cloud Storage bill is $50K/month. Implement lifecycle policies to reduce costs.

Step 1: Analyze Current Usage

# Get bucket sizes by storage class
gsutil du -s gs://data-lake-prod/

# Analyze access patterns with Cloud Monitoring
gcloud logging read \
  'resource.type="gcs_bucket" AND
   protoPayload.methodName:"storage.objects.get"' \
  --format="csv(timestamp,protoPayload.resourceName)" \
  --limit=10000 > access_log.csv

# Check bucket metadata
gcloud storage buckets describe gs://data-lake-prod --format=yaml

Step 2: Understand Storage Class Economics

Step 3: Implement Lifecycle Policies

// lifecycle-data-lake.json
{
  "lifecycle": {
    "rule": [
      {
        "action": {
          "type": "SetStorageClass",
          "storageClass": "NEARLINE"
        },
        "condition": {
          "age": 7,
          "matchesPrefix": ["raw/"]
        }
      },
      {
        "action": {
          "type": "SetStorageClass",
          "storageClass": "COLDLINE"
        },
        "condition": {
          "age": 30,
          "matchesPrefix": ["raw/"]
        }
      },
      {
        "action": {
          "type": "SetStorageClass",
          "storageClass": "ARCHIVE"
        },
        "condition": {
          "age": 90,
          "matchesPrefix": ["raw/"]
        }
      },
      {
        "action": {
          "type": "SetStorageClass",
          "storageClass": "NEARLINE"
        },
        "condition": {
          "age": 30,
          "matchesPrefix": ["processed/"]
        }
      },
      {
        "action": {
          "type": "SetStorageClass",
          "storageClass": "ARCHIVE"
        },
        "condition": {
          "age": 0,
          "matchesPrefix": ["archives/"]
        }
      },
      {
        "action": {
          "type": "Delete"
        },
        "condition": {
          "age": 2555,
          "matchesPrefix": ["raw/"]
        }
      }
    ]
  }
}

# Apply lifecycle policy
gcloud storage buckets update gs://data-lake-prod \
  --lifecycle-file=lifecycle-data-lake.json

Step 4: Terraform Implementation

resource "google_storage_bucket" "data_lake" {
  name     = "data-lake-prod"
  location = "US"

  # Enable uniform bucket-level access
  uniform_bucket_level_access = true

  # Lifecycle rules for cost optimization
  lifecycle_rule {
    condition {
      age                   = 7
      matches_prefix        = ["raw/"]
    }
    action {
      type          = "SetStorageClass"
      storage_class = "NEARLINE"
    }
  }

  lifecycle_rule {
    condition {
      age                   = 30
      matches_prefix        = ["raw/"]
    }
    action {
      type          = "SetStorageClass"
      storage_class = "COLDLINE"
    }
  }

  lifecycle_rule {
    condition {
      age                   = 90
      matches_prefix        = ["raw/"]
    }
    action {
      type          = "SetStorageClass"
      storage_class = "ARCHIVE"
    }
  }

  # Delete raw data after 7 years (compliance requirement)
  lifecycle_rule {
    condition {
      age                   = 2555
      matches_prefix        = ["raw/"]
    }
    action {
      type = "Delete"
    }
  }

  # Processed data to Nearline after 30 days
  lifecycle_rule {
    condition {
      age            = 30
      matches_prefix = ["processed/"]
    }
    action {
      type          = "SetStorageClass"
      storage_class = "NEARLINE"
    }
  }

  # Versioning with cleanup
  versioning {
    enabled = true
  }

  lifecycle_rule {
    condition {
      num_newer_versions = 3
      with_state         = "ARCHIVED"
    }
    action {
      type = "Delete"
    }
  }
}

Step 5: Use Autoclass for Unpredictable Access

# For buckets with unpredictable access patterns
resource "google_storage_bucket" "user_uploads" {
  name     = "user-uploads-prod"
  location = "US"

  # Autoclass automatically moves objects between classes
  # based on access patterns
  autoclass {
    enabled = true
  }

  # You cannot use lifecycle rules with Autoclass
  # Autoclass handles transitions automatically
}

# Enable Autoclass on existing bucket
gcloud storage buckets update gs://user-uploads \
  --enable-autoclass

Step 6: Optimize Operations Costs

# Current: 500M Class A ($2,500), 2B Class B ($800)

# Reduce operations:
# 1. Use composite objects for small files
gsutil compose gs://bucket/part1 gs://bucket/part2 gs://bucket/combined

# 2. Batch requests
from google.cloud import storage
client = storage.Client()
bucket = client.bucket('data-lake-prod')

# Batch list operations
blobs = list(bucket.list_blobs(prefix='raw/', max_results=1000))

# 3. Use Cloud CDN for frequently accessed objects
# Caches at edge, reduces origin requests

Step 7: Cost Calculation

BEFORE:
- Standard storage: 55TB × $0.020 = $1,100/month
- Operations: $3,300/month
- Egress + other: ~$45,600/month
- Total: ~$50,000/month

AFTER (projected):
Data Lake (45TB):
- raw/ 25TB:
  - 7 days Standard: 0.6TB × $0.020 = $12
  - 23 days Nearline: 1.9TB × $0.010 = $19
  - Coldline: 7TB × $0.004 = $28
  - Archive: 15.5TB × $0.0012 = $19
- processed/ 15TB Nearline: 15TB × $0.010 = $150
- archives/ 5TB Archive: 5TB × $0.0012 = $6
Subtotal: ~$234/month (was ~$900)

User Uploads (10TB with Autoclass):
- ~$150/month (was ~$200)

Operations (reduced with batching):
- ~$2,000/month (was $3,300)

New Total: ~$20,000/month (60% reduction)

Step 8: Monitor and Validate

# Alert on unexpected storage class changes
resource "google_monitoring_alert_policy" "storage_class_change" {
  display_name = "Unexpected Storage Class Distribution"

  conditions {
    display_name = "Standard storage exceeds threshold"

    condition_threshold {
      filter = <<-EOT
        resource.type="gcs_bucket" AND
        metric.type="storage.googleapis.com/storage/total_bytes" AND
        metric.labels.storage_class="STANDARD"
      EOT

      comparison      = "COMPARISON_GT"
      threshold_value = 10000000000000  # 10TB
      duration        = "3600s"

      aggregations {
        alignment_period   = "3600s"
        per_series_aligner = "ALIGN_MEAN"
      }
    }
  }

  notification_channels = [google_monitoring_notification_channel.email.id]
}