A container's memory usage keeps growing until it gets OOM killed. How do you diagnose and fix it?

Step 1: Confirm It’s Actually a Leak

# Monitor memory over time
docker stats api-service

# Export metrics to analyze trend
docker stats --format "{{.MemUsage}}" api-service >> memory.log

# Check if memory grows continuously or reaches a plateau
watch -n 5 'docker stats --no-stream api-service'

Memory growth patterns:

• Leak: Continuous unbounded growth until OOM
• Cache: Growth to a plateau, then stable
• Normal: Growth during load, decrease after

Step 2: Profile Memory Inside Container

# Get shell access
docker exec -it api-service sh

# For Node.js - check heap usage
$ node -e "console.log(process.memoryUsage())"

# For Python - check memory
$ python -c "import resource; print(resource.getrusage(resource.RUSAGE_SELF).ru_maxrss)"

# For any process - use top
$ top -b -n 1 | head -20

# Check process memory details
$ cat /proc/1/status | grep -i mem
VmRSS:	  524288 kB  # Resident memory
VmSize:	 1048576 kB  # Virtual memory

Step 3: Identify Leak Source (Node.js Example)

# Add debugging flags to CMD
CMD ["node", "--expose-gc", "--max-old-space-size=400", "server.js"]

// Add memory monitoring endpoint
app.get('/debug/memory', (req, res) => {
  const used = process.memoryUsage();
  res.json({
    heapTotal: `${Math.round(used.heapTotal / 1024 / 1024)} MB`,
    heapUsed: `${Math.round(used.heapUsed / 1024 / 1024)} MB`,
    external: `${Math.round(used.external / 1024 / 1024)} MB`,
    rss: `${Math.round(used.rss / 1024 / 1024)} MB`
  });
});

// Force garbage collection endpoint (dev only)
app.post('/debug/gc', (req, res) => {
  if (global.gc) {
    global.gc();
    res.json({ status: 'GC triggered' });
  } else {
    res.status(400).json({ error: 'GC not exposed. Run with --expose-gc' });
  }
});

Step 4: Common Leak Patterns and Fixes

Event Listeners Not Removed:

// LEAK: Adding listener on every request
app.get('/data', (req, res) => {
  eventEmitter.on('data', handler); // Never removed!
});

// FIX: Use once or remove listener
app.get('/data', (req, res) => {
  eventEmitter.once('data', handler);
  // OR
  const handler = (data) => { /* ... */ };
  eventEmitter.on('data', handler);
  req.on('close', () => eventEmitter.off('data', handler));
});

Unclosed Database Connections:

// LEAK: Connections never returned to pool
async function query(sql) {
  const conn = await pool.getConnection();
  const result = await conn.query(sql);
  // Connection never released!
  return result;
}

// FIX: Always release connections
async function query(sql) {
  const conn = await pool.getConnection();
  try {
    return await conn.query(sql);
  } finally {
    conn.release();
  }
}

Growing Caches:

// LEAK: Unbounded cache
const cache = {};
function getCached(key) {
  if (!cache[key]) {
    cache[key] = expensiveComputation(key);
  }
  return cache[key];
}

// FIX: Use LRU cache with max size
const LRU = require('lru-cache');
const cache = new LRU({ max: 500 });