← Dev log

devops

PM2 cluster mode, and the state that breaks when you turn it on

PM2 cluster mode, and the state that breaks when you turn it on

Node runs JavaScript on a single thread. I/O offloads to a background pool so the main thread isn't blocked, but actual execution — parsing a request, running a handler, serialising a response — happens on that one thread, one operation at a time. A 32-core server running one node dist/main.js gets the same single-core ceiling as a 2-core laptop. It's not a missing config flag; it's the runtime's design.

I found this the concrete way: a load test where docker stats during the run showed one core pegged near 100% and nineteen idle.

module.exports = {
  apps: [{
    script: "dist/src/main.js",
    exec_mode: "cluster",
    instances: 2,
    autorestart: true,
    max_restarts: 10,
    max_memory_restart: "450M",
  }],
};

PM2 forks N workers all binding the same port, with the OS load-balancing across them. On top of that, for near-zero extra code: auto-restart on crash, zero-downtime rolling reload, log aggregation into one stream. Compared to Node's raw cluster module, PM2 removes the fork and master-worker bookkeeping entirely.

Turning it on isn't automatically safe

The moment there's more than one process, anything held in one process's memory stops being consistent — a client's requests can land on different workers that know nothing about each other. Three things break silently:

  • In-memory rate limiters — each worker gets its own counter, and an N-worker cluster silently multiplies every limit by N.
  • In-memory Socket.io room state — a client on worker 1 who should get a broadcast from worker 2 simply won't.
  • Raw setInterval background jobs guarded only by an in-process flag — N workers means N-way duplicate processing unless there's a real atomic claim coordinating across them.

The fix for all three is the same shape: move the shared state somewhere every process can see. Redis is the default — atomic INCR for counters, a pub/sub adapter for Socket.io, SET NX for locks. Do this before flipping on multiple instances. (When I audited this, two of my three background jobs turned out to be missing the atomic claim.)

Not Kubernetes

PM2 clustering scales within one machine, up to its core count. Kubernetes scales across machines, with rolling deploys and self-healing pods on top. Reaching for Kubernetes when the actual constraint is "one process can't use more than one core" is solving a problem you don't have yet, at the cost of a control plane and manifests you now operate. The right trigger for Kubernetes is having maxed out clustering on the best single server available and still needing more.

Proving it

exec_mode: "cluster" proves nothing on its own. What actually proved it: pm2 list showing both workers independently online; SIGKILL-ing both worker PIDs and watching PM2 respawn them within seconds; and — the one that validates the shared-state assumption — running a deliberately low rate limit across both workers and confirming exactly N successes, not 2N.